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ABSTRACT 


Two  methods  of  inspecting  pairs  of  comparison  photographs  were  studied:  1) 
side-by-side  display;  and  2)  apparent-motion  display,  in  which  pictures  in 
spatial  registry  were  presented  in  temporal  alternation.  This  second  presenta¬ 
tion  method  produces  apparent  motion,  where  a  difference  between  the  pictures 
exists,  which  aids  in  the  rapid  detection  of  objects  that  have  changed. 

Both  experienced  anu  inexperienced  interpreters  were  tested  with  aerial  photo¬ 
graphy  varying  in  scale,  contrast,  resolution,  and  terrain  complexity. 

Under  certain  conditions  the  apparent-motion  display  method  was  found  to 
enhance  significantly  interpreter  performance  in  the  change  detection  task.  How¬ 
ever,  it  became  relatively  less  effective  with  poorer  quality  imagery  or  where 
these  was  a  high  percentage  of  irrelevant  change.  Under  difficult  conditions, 
neither  display  method  was  significantly  better  than  the  olher. 

Although  the  applications  of  the  apparent-motion  technique  are  limited,  the 
results  of  this  study  suggest  the  value  of  further  efforts  to  define  the  extent  of 
its  usefulness. 
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INTRODUCTION 


THE  PROBLEM:  INCREASING  DEMAND  FOR  RECONNAISSANCE 

Since  the  development  of  military  aircraft,  each  major  conflict  among  nations 
has  produced  heavy  demands  for  aerial-reconnaissance  information.  The 
Russian  buildup  of  missiles  in  Cuba  propelled  aerial  reconnaissance  into  the 
limelight  as  one  of  the  nation's  first  lines  of  information. 

According  to  John  McCone,  Chief  of  the  Central  Intelligence  Agency: 

■'Every  war  of  this  century,  including  World  War  I,  has  started  because 
of  inadequate  intelligence  and  incorrect  intelligence  estimates  and  eval¬ 
uations.  This  was  true  of  Pearl  Harbor,  for  example,  and  it  was  true 
in  Korea.  The  Cuban  crisis  in  October  could  have  generated  a  war,  some 
think  a  nuclear  war.  But  war  over  Cuba  was  avoided  because  of  intelli¬ 
gence  success.  Every  threat  to  our  security,  every  weapons  system, 
was  correctly  identified  in  time  to  give  the  President  and  his  policy 
advisers  time  to  think,  to  make  a  rational  estimate  of  the  situation,  and 
to  devise  a  means  of  dealing  with  it  with  a  maximum  chance  of  success 
and  a  minimum  risk  of  global  war.  I  consider  this  an  intelligence  suc¬ 
cess.  Although  intelligence  is  not  a  measurable  commodity,  that  is  at 
least  a  partial  measure  of  its  value.1'  (Alsop,  1963) 

THE  BOTTLENECK:  INFORMATION  EXTRACTION 

Historically,  the  reconnaissance  system  has  always  been  an  ancillary  system; 
weapon  systems  have  received  the  higher  priorities  in  design  and  development 
of  equipment,  manufacture,  and  manpower.  In  addition  to  this  secondary  role, 
there  has  been  a  developmental  imbalance  within  the  reconnaissance  system. 

The  development  of  airborne  and  space-observation  platforms,  reconnaissance 
sensors,  and  methods  and  equipment  for  storage  and  retrieval  has  moved  ahead 
with  rapid  technological  sLrides.  In  contrast,  almost  completely  neglected  were 
the  scientific  investigations  of  three  man-machine  interface  areas  associated 
with  information  extraction:  (l)  classifying  the  usefulness  of  the  pictures, 

(2)  encoding  pictures  for  easy  retrieval,  and  (3)  interpreting  (i.e,  ,  transforming 
pictures  into  linguistic,  digital,  or  analog  forms  of  information  and  relating 
these  new  data  to  previous  information). 

The  flow  of  information  through  the?  reconnaissance  system  is  no  faster  than  its 
least  efficient  subsystem.  The  amount  of  intelligence  data  that  can  pass  through 
our  limited  capabilities  of  classifying,  encoding,  and  interpreting  pictures  cannot 
be  increased  by  greater  expenditure  of  money  and  time  on  the  already  advanced 
technologies  of  platforms,  sensors,  and  storage  devices. 


Photointerpreters  must  work  in  all  three  areas  of  classifying,  encoding,  and 
interpreting  pictures.  Since  their  skills  are  most  needed  in  the  interpretation  of 
aerial  photography  and  in  the  transformation  of  images  into  meaningful  terms, 
efficiency  would  demand  that  the  photographs  placed  before  them  would  be  pre¬ 
selected  for  good  quality,  coded  for  information  content  and  image  characteristics, 
identified  by  geographic  location,  and  indexed  by  areas  that  have  a  high  probability 
of  containing  information. 

The  requirement  for  global  surveillance,  wherein  pictures  from  space  and  from 
the  atmosphere  would  be  used  to  keep  all  nations  well  informed  as  to  the  activities 
of  each,  greatly  increases  the  problem  of  information  extraction.  On  the  basis  of 
of  the  photographic  output  of  the  earliest  weather  satellite  and  the  published 
information  about  our  military  needs,  we  could  collect  enough  pictures  to  keep 
approximately  50,000  photointerpreters  busy.  (Kraft  and  Hamilton,  1961;  Kraft 
and  Klingberg,  1962,)  The  number  of  trained  and  active  photointerpreters  is 
far  less  than  50,000.  However,  current  operational  practice  operates  on  the 
philosophy  that  the  photographic  interpreter  is  the  "one  who  knows  what  he  is 
looking  for,”  a  philosophy  that  requires  his  participation  in  all  data-transformation 
phases. 

The  limited  number  of  trained  photointerpreters  and  their  widespread  use  within 
the  system  combine  to  make  the  man-machine  interfaces  the  critical  problem 
areas  of  the  reconnaissance  system. 

APPROACHES  TO  SOLVING  THE  PROBLEM 

The  most  frequently  suggested  solution  is  to  build  automatic  machines  to  do  the 
classification,  encoding,  and  interpretation  jobs.  However,  complete  automa¬ 
tion  is  not  the  immediate  answer.  If  and  when  completely  automatic  equipment 
is  design  and  built,  its  complexity  and  its  associated  maintenance  cost  will  have 
to  be  compared  with  its  effectiveness. 

A  second  solution  is  to  increase  the  number  of  trained  photo  interpreters.  To 
maintain  a  staff  of  50,000  or  more  trained  photoinLerp rotors  would  require  an 
entirely  new  selection  and  training  program,  and  a  special  military  or  civil - 
service  job  category.  The  latter  would  be  necessary  to  provide  economic  and 
community  status  inducements  to  maintain  long-term  job  satisfaction  so  that  the 
individual's  training  and  experience  would  remain  available  to  the  using  agencies. 

A  third  alternate  appears  to  be  the  most  promising  and  immediately  applicable. 

This  alternate  is  the  scientific  study  of  man  and  machine  performance  within  the 
reconnaissance  system  and  the  application  of  these  evaluations  to  improve  the 
performance  of  the  semiautomatic  system.  The  goal  would  be  the  optimum 
design  of  equipment,  use  of  procedures,  assignment  of  tasks,  and  organization 
of  work  environment  against  a  criterion  of  maximum  man  and  machine  perform¬ 
ance  in  processing  of  reconnaissance  information. 


The  overall  problem  is  too  large  to  be  handled  at  one  time  by  any  single  organi¬ 
zation,  but  a  beginning  is  to  determine  if  the  design  or  redesign  of  display 
equipment  would  permit  large  numbers  of  briefly  trained  persons  to  take  over 
part  of  the  interpreter's  task.  This  would  be  feasible  only  if  it  could  be  demon¬ 
strated  that  the  briefly  trained  individual  could,  with  a  special  display,  perform 
as  efficiently  as  a  fully  trained  interpreter.  System  performance  could  then  be 
improved  by  having  the  experienced  photointerpreter  apply  himself  more  to 
technical  interpreting — the  task  most  demanding  of  his  training  and  experience. 

One  of  the  important  tasks  of  the  pholointerpretcr  is  to  detect  meaningful  changes 
in  comparative-cover  photography.  Comparative-cover  photography  consists  of 
two  or  more  images  of  the  same  terrain  obtained  at  different  times.  Comparison 
of  the  two  photographic  samples  then  provides  difference  information  on  changes 
that  might  have  occurred  in  the  elapsed  time.  The  differences  or  changes,  if 
relevant  to  man's  activity,  are  then  the  sources  of  information  that  provide 
intelligence  about  his  activities.  World  War  II  provided  examples  that  amplified 
the  importance  of  this  comparative  technique.  One  such  example  is  the  photo¬ 
graph  of  the  northern  Axis  ports  obtained  by  a  British  Spitfire  3  days  before  the 
Norway  invasion.  Because  of  the  absence  of  a  comparative  standard,  the  ports’ 
invasion  preparations  were  interpreted  as  "normal  port  activity. "  On  the  other 
hand,  an  excellent  example  of  the  usefulness  of  the  technique  occurred  when  the 
Allied  powers  were  able  to  observe  the  progress  of  the  V-l  buildup  through  the 
comparison  of  successive  photographs.  By  bombing  the  sites  just  before  their 
proposed  initial  use,  they  significantly  delayed  use  of  the  V-l  as  an  operational 
weapon. 


METHODS  OF  DISPLAYING  COMPARATIVE -COVER  PHOTOGRAPHS 


The  availability  of  two  photographs  of  the  same  terrain  makes  possible  direct 
comparison.  Experimental  evidence  shows  man  to  be  far  more  effective  in 
making  comparative  discriminations  than  in  making  absolute  (no  physical 
standard)  discriminations.  The  efficiency  of  methods  of  displaying  compara¬ 
tive-cover  photographs  might  therefore  vary  as  a  function  of  the  degree  to 
which  they  provide  ideal  conditions  for  comparative  judgments. 

The  visual-perceptual  elements  of  the  change-discrimination  task  (rather  than 
the  cognitive  elements  of  the  task)  appear  to  be  primary,  and  display  designs 
optimizing  the  conditions  for  visual  discrimination  might:  (1)  improve  the  per¬ 
formance  of  both  trained  and  untrained  observers,  and  (2)  for  this  task,  permit 
the  performance  of  the  non-photointerpreter  to  be  comparable  to  that  of  the 
trained  interpreter.  If  research  findings  established  the  validity  of  the  latter  or 
both  suppositions,  a  division  of  tasks  would  improve  the  efficiency  of  the  system. 
The  tasks  that  are  more  dependent  on  the  visual-perceptual  elements  could  be 
assigned  to  quickly  trained  personnel  using  specially  designee  equipment,  and 
the  tasks  that  include  principally  cognitive  elements  would  be  reserved  for  the 
photointerpreter. 

The  initial  step  in  the  study  of  using  man-machine  optimization  was  to  ascertain 
what  visual  information  should  be  attenuated,  what  information  should  be  enhanced, 
and  which  display  methods  would  be  operationally  feasible.  Changes  in  number, 
size,  position,  color,  and  configuration  must  be  enhanced  by  the  display  system. 
Identical  areas  in  both  photographs  should  be  attenuated  by  the  display  system. 
Another  dimension  of  the  problem  is  that  changes  can  be  either  relevant  or  irrele¬ 
vant.  Relevant  changes  are  those  that  tell  of  man’s  activity  or  of  conditions  of 
nature  that  would  affect  his  activity.  Natural  changes  that  do  not  significantly 
alter  man’s  activity  are  considered  irrelevant  to  the  requirements  of  the  intelli¬ 
gence  organizations.  The  enhancement  of  all  changes,  relevant  and  irrelevant, 
might  not  increase  the  flow  of  information:  large  numbers  of  enhanced  irrelevant 
changes  might  even  reduce  it.  For  example,  a  large  difference  in  sun  angle  be¬ 
tween  the  two  pictures  will  result  in  an  increase  in  the  amount  of  irrelevant  change 
as  a  function  of  the  number  of  tall  objects  in  the  scene.  Therefore,  the  display, 
in  addition  to  enhancing  the  changes  and  attenuating  the  nonchanged  area,  must 
permit  differentiation  into  relevant  and  irrelevant  changes.  Accordingly,  the 
ideal  display  of  reconnaissance  photography  should  result  in  the  transmittal  of 
information  to  intelligence  as  it  is  represented  in  Table  I, 


Pr© 


Table  I  INFORMATION  TRANSMITTAL  FROM  AN  IDEAL 
CHANGE-DETECTION  DISPLAY 


RELEVANT 

RELEVANCE  TO 

INTELLIGENCE 

ORGANIZATIONS 

IRRELEVANT 


CHANGES  IN  THE  PHOTOGRAPH 


NONE 

SOME 

No  Change 

Detection, 

Recognition  and 

Recognition, 

No  Report 

and  Report  ^ 

No  Change 

Detection, 

Recognition  and 

Recognition,  and 

No  Report 

No  Report 

The  ideal  display  should  maximize  the  conditions  leading  to  reports  of  change 
associated  with  the  Box  B  in  Table  I.  Errors  of  omission  arc  a  product  of  a 
nonreport  from  Box  B,  and  such  errors  would  decrease  the  measure  of  complete¬ 
ness.  The  ideal  display  should  facilitate  the  recognition  of  no  change  in  Boxes  A 
and  C,  a  condition  that  would  be  reflected  in  lower  time  scores.  Reports  of 
changes  from  Box  D  are  errors  of  commission  that  reflect  inability  to  determine 
the  relevance  of  the  change.  Reports  of  changes  from  Boxes  A  and  C  would  re¬ 
flect  errors  of  commission  based  on  an  inability  to  discriminate  unchanged  areas 
from  changed  areas. 

SIDE-BY-SIDE  METHOD 

The  comparison  of  two  photographs,  without  specialized  equipment,  is  made  by 
placing  the  two  prints  side  by  side  and  looking  from  one  to  the  other  in  making 
the  visual  analysis.  The  simplicity  and  directness  of  this  method  had  made  it 
the  operational  standard,  especially  in  the  field.  The  interpreter's  aids  for 
this  method  are  magnifiers,  light  tables,  measuring  de\  ices,  and  interpreter 
keys.  Essentially,  the  method  of  side-by-side  comparison  is  unchanged  by  the 
addition  of  these  aids.  This  method  was  chosen  as  the  basis  for  comparison 
with  other  display  methods. 

OVERLAY  METHOD 

One  experimental  approach  involves  the  attenuation  of  the  brightnesses  of  all 
unchanged  areas.  This  might  be  accomplished  by  sandwiching  negative  and 
positive  photographic  transparencies  in  registry  in  a  single  optical  light  path. 
Changes  should  appear  lighter  or  darker  against  the  gray  of  equal  attenuation 
by  the  negative -positive  subtraction. 


Perfect  registry  of  the  two  sets  of  images  prevents  the  observer  from  recog¬ 
nizing  the  shapes  of  objects  and  therefore  interferes  with  the  discrimination 
of  relevancy  of  the  change.  Slight  misregistry  will  produce  outline  figures,  as 
in  Figure  1,  when  the  stimuli  are  simple  forms,  and  a  bas-relief  effect  with 
more  complex  stimuli  as  shown  in  Figure  IB.  This  permits  some  degree  of 
form  recognition  and  discrimination  of  relevance. 

A  variation  of  the  overlay  method  is  to  place  the  negative  transparency  in  one 
optical  pathway  and  the  positive  in  a  second  pathway,  combining  them  with  a 
two-way  mirror  so  the  ubserver  sees  them  in  slight  offset  registry.  The  two 
pictures  combined  in  this  way  produce  a  "silvery"  appearance  for  simple  forms. 
This  appearance  has  been  called  Titchener’s  "scintillation  effect.  "  His  early 
handbook  on  experimental  laboratory  methods  included  five  stereoscopic  slides 
that  produced  this  phenomenon  (Titchener,  1901  and  1915).  The  effect  is  pro¬ 
duced  by  stereograms  with  a  black-line  figure  on  a  white  field  (square,  rectangle, 
or  three-dimensional  drawing)  presented  to  one  eye  and  a  white -line  drawing 
(also  on  a  white  field)  presented  to  the  other  eye  but  with  the  internal  area  of  the 
figure  black  in  color.  This  internal  area  common  to  the  two  figures  will  be  seen 
as  "silvery"  when  second-degree  fusion  occurs  for  the  observer  viewing  the  card 
in  a  stereoscope. 

In  the  studies  reported  here,  the  overlay  method  used  two  optical  beams  with  a 
slight  offset  in  the  alignment  of  positive  and  negative  transparencies. 

APPAHENT-MOTION  METHOD 

The  apparent-motion  method,  as  used  in  these  investigations,  consisted  of  placing 
a  positive  transparency  in  one  optical  pathway,  the  comparative  positive  trans¬ 
parency  in  the  second  pathway,  and  combining  them  with  a  two-way  mirror  so 
that  the  observer  saw  them  in  registry.  The  two  pathways  were  illuminated 
alternately  in  as  near  an  approximation  to  a  square  wave  pattern  as  the  mechanical 
shutter  permitted.  Two  identical  and  registered  pictures,  combined  in  this  way, 
produce  the  appearance  of  a  single  continuous  picture.  Nonidentical  areas  of  the 
two  transparencies  appear  to  move  since  the  alternation  rate  of  1.5  cycles  per 
second  is  within  the  range  in  which  apparent  motion  is  perceived. 

The  use  of  this  phenomenon  in  developing  a  display  that  would  enhance  change 
detection  is  based  on  two  assumptions: 

1)  The  detection  threshold  for  a  moving  (changing)  object  would  be  lower  than 
that  for  stationary  objects.  As  early  as  1878,  G,  H.  Schneider  (Boring, 

1942)  had  determined  that  a  moving  shadow  is  more  perceptible  than  a  still 
shadow. 

2)  In  most  instances,  the  number  of  moving  objects  would  be  a  smaller  portion 
of  the  number  of  objects  in  the  field  of  view.  Thus,  the  changed  object 
would  contrast  with  the  surrounding  stationary  objects. 
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Figure  1 

BAS-RELIEF  EFFECT  IN  STIMULI  OF  LOW  COMPLEXITY  (A), 
AND  HIGH  COMPLEXITY  AND  LARGE  DISPLAY  SCALE  (B). 
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The  history  of  apparent  motion,  as  distinguished  from  the  perception  of  real 
motion,  began  in  1820  with  Purkinje’s  phenomenological  description  of  seen 
and  felt  motion  in  giddiness  (Boring,  1942).  Apparent  motion,  free  of  nystag¬ 
mic  eye  movements,  was  described  by  Roget  in  1825  on  noting  the  stationary 
appearance  of  a  moving  carriage  wheel  when  viewed  through  a  picket  fence. 

In  the  following  10  years.  Plateau  and  Stampfer  independently  developed  the 
concept  and  the  device  known  as  the  stroboscope.  The  earliest  thorough  dis¬ 
cussion  of  the  phenomenon  is  to  be  found  in  HeLmholtz's  Handbuch  der  Physiolo- 
gisehen  Qptik  (I860),  and  Thomas  A.  Edison  in  1894  made  a  practical  use  of 
the  phenomenon  in  the  development  of  motion  pictures.  Although  Edison’s 
kinetoscope  started  a  series  of  applications  such  as  the  "moving”  billboards, 
railroad-crossing  signals,  and  Times  Square  "newspaper  in  the  sky,  "it  was 
Wertheimer  (1912)  who  simplified  the  observational  situation,  presenting  a 
single  discrete  displacement  of  a  form  with  a  tachistoscope.  As  he  varied  the 
time  interval  between  the  presentations  of  these  two  forms  between  0.  030  and 
0.200  second,  he  observed  a  series  of  different  kinds  of  apparent  movement. 

In  order  of  increasing  time  intervals,  he  reported  simultaneity,  optimum  move¬ 
ment,  partial  movement,  pure  movement,  and  succession.  Wertheimer’s  expla¬ 
nation  of  what  occurred,  with  the  right  time  intervals  to  produce  pure  movement, 
was  that  the  seen  movement  was  like  a  "physiological  short-circuit"  in  the  brain 
—  a  cortical  process  that  is  the  physiological  substrate  of  apparent  motion. 
Wertheimer’s  paper,  with  the  then-current  enthusiasm  for  Gestalt  psychology, 
produced  such  interest  that  more  than  a  hundred  papers  on  apparent  movement 
appeared  in  the  next  30  years.  Among  these  were  three  that  isolated  and  named 
six  types  of  apparent  motion  besides  that  which  Wertheimer  called  phi  move¬ 
ment.  These  types  of  motion  all  would  be  seen  in  discriminating  changes  in 
photographs.  These  types  of  changes  arc  listed  here  by  the  names  the  authors 
gave  them  but  defined  in  terms  of  their  application  to  comparing  aerial  photo¬ 
graphs  for  change. 


Beta  movement  (Kenkel,  1913):  An  example  would  be  that  of  two  photo grapi is 
taken  1  minute  apart  of  an  army  tank  in  motion.  The  two  pictures  would 
show  the  tank  in  two  different  locations  along  the  line  of  its  track.  When 
viewed  in  the  apparent-motion  equipment,  the  tank  would  appear  to  be 
jumping  back  and  forth  between  the  two  photographed  positions. 

Alpha  movement  (Kenkel,  1913):  If  a  long  building  had  been  extended  in 
the  time  between  the  two  comparative  coverages,  in  the  apparent-motion 
device  the  observer  would  sec  the  building  alternately  lengthen  and  shorten. 

Gamma  movement  (Kenkel,  1913);  If  an  oil  tank  in  a  tank  farm  had  been 
replaced  by  a  larger  tank,  its  appearance  would  be  as  for  alpha  movement, 
a  change  in  size;  but  the  gamma  motion  is  like  a  swelling  and  shrinking,  or 
a  motion  along  the  line  of  sight  toward  the  observer.  Gamma  movement 
may  also  be  seen  where  there  is  a  brightness  difference  in  an  ambiguously 
defined  area  such  as  specular  reflection  on  water. 
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Delta  movement  (Korte,  1915):  A  reversed  movement  from  the  order  of 
presentation  due  to  the  relative  brightness.  A  motion  that  might  be  per¬ 
ceived  but  not  differentiated  from  gamma  in  this  display  due  to  the  presen¬ 
tation  order  of  the  photos  not  being  known  to  the  observer. 

Bow  movement  (Benussi,  1916):  A  curved  movement  that  does  not  follow 
the  shortest  distance  between  two  points.  An  example  would  be  a  portable 
airplane -repair  platform  that  had  been  moved  to  the  other  side  of  the 
fuselage  between  photos.  This  would  appear  to  move  with  a  curved  course 
over  the  airplane. 

Split  movement  (De  Silva,  1926):  Such  motion  may  be  ambiguous,  such  as 
a  single  packing  case  seen  in  one  photograph,  but  between  two  other  crates 
in  the  second  photograph.  The  apparent  motion  appears  to  originate  with 
the  single  crate  and  go  two  directions  toward  the  two  new  crates. 

Another  visual  phenomenon,  although  not  true  apparent  motion,  is  impor¬ 
tant  in  discriminating  changes  in  photographs.  If  an  object  is  removed  or 
added,  and  no  similarly  shaped  or  sized  object  is  nearby,  the  added  object 
produces  a  flicker. 

Although  all  these  classifications  of  apparent  motion  are  seen  by  people  in  the 
course  of  using  the  apparent-motion  display  apparatus,  knowledge  about  types 
and  extent  of  motion  is  not  essential  to  the  discrimination  of  change.  The  types 
of  motion,  on  the  other  hand,  could  be  developed  as  aids  in  determining  what 
change  has  occurred  and  its  relevancy, 

EARLY  RESEARCH  IN  THIS  LABORATORY 

The  three  display  techniques  reviewed  above — side-by-side,  overlay,  and 
apparent  motion  — have  been  used  in  a  series  of  experiments  on  change  detec¬ 
tion  in  comparative-cover  photographs. 

Before  these  systematic  experiments,  a  modified  Wheatstone  stereoscope  was 
used  to  investigate  change  detection  with  the  apparent-motion  technique  (Larry, 
1960).  With  this  equipment,  it  was  possible  to  vary  the  rate  of  alternation, 
provide  u  steady  illuminance  to  one  beam  and  pulse  the  other,  or  have  two  steady 
illuminances.  Writh  the  stereoscopic  display,  one  eye  sees  the  earlier  photo¬ 
graphic  sample  and  the  other  eye  is  presented  with  the  new  sample.  However, 
the  steroscopic  approach  was  not  used  in  the  series  of  studies  that  follow  for 
these  technical  reasons: 

1)  Stereoscopic  viewing  makes  second-degree  fusion  almost  impossible  for 
subjects  with  any  degree  of  convergence-accommodation  imbalance  when  the 
objects  viewed  are  a  negative  to  one  eye  and  a  positive  to  the  other  eye. 

2)  The  apparatus  is  not  easily  adapted  to  all  three  display  conditions  — 
side-by-side,  overlay,  and  apparent  movement. 
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3)  The  interobser\ er  variability  would  be  excessive  unless  subjects  were 
selected  on  the  basis  of  ideal  visual  skills. 

A  second  instrument  was  designed  that  would  not  have  these  limitations. 

Figure  2  is  a  functional  diagram  of  the  apparatus.  The  illuminance  for  the  two 
optical  pathways  for  the  photographic  slides  emanates  from  a  single  source. 

The  tungsten  lamp  was  housed  in  an  air-cooled  aluminum  sphere  drilled  to 
provide  two  point  sources,  one  for  each  optical  path.  A  variable-speed  motor 
powered  the  conical  shutter  that  occluded  the  beams  alternately.  The  shutter 
could  be  set  in  a  fixed  position  and  a  second  port  opened  permitting  the  simul¬ 
taneous  passage  of  both  beams.  The  light  emanating  from  the  ports  in  the 
sphere  was  spread  by  the  biconcave  lenses  to  provide  a  large  illuminated  area 
on  the  opal  screens.  These  illuminated  opal  screens  were  needed  as  area 
sources  because  the  focusing  of  the  point  sources  by  the  Fresnel  lens  resulted 
in  an  interference  spectrum  at  the  eyepiece.  The  thin  plastic  Fresnel  lenses 
(14 -inch  focal  length,  14 -inch  diameter)  were  mounted  in  16 -inch  by  lG-inch 
heavy  aluminum  clamping  frames.  The  large -Hi  a  meter  Fresnel  lenses  allowed 
the  side-by-side  presentation  of  two  4-inch  by  4 -inch  photographic  transparen¬ 
cies  within  an  equal-brightness  humogeneous  field.  The  slide  holders  were 
built  so  that  the  transparencies  could  be  positioned  in  the  center  for  apparent- 
motion  and  overlay  viewing,  or  to  one  side  for  side-by-side  viewing.  The  slide 
holders  were  not  adjustable  for  individual  slide-registration  adjustments;  slides 
are  registered  when  mounted.  The  two-way  mirror,  mounted  45  degrees  to  the 
observer's  line  of  sight,  permitted  simultaneous  or  alternate  viewing  of  both 
pathways.  The  slides  were  centered  when  in  registry. 

The  third  optical  path  for  the  coding  matrix  was  also  registered  with  the  centered 
slides.  In  side-by-side  presentations,  the  coding  matrix  was  positioned  to 
register  with  the  left  side.  The  coding  matrix  has  two  illumination  sources  that 
operate  independently.  The  observer  activates  the  system  by  keying  a  lattice 
matrix  of  3G  squares  ordered  in  six  rows  and  six  columns.  This  matrix  is 
formed  by  painLing  an  electro -luminescent  panel  to  black  out  all  areas  but  the 
lines  forming  the  matrix.  Centered  within  each  square  of  this  matrix  is  a  5/8- 
inch  hole  punched  through  the  electroluminescent  material.  The  electrolumines¬ 
cent  panel  is  mounted  as  the  front  face  of  a  small  cabinet  containing  thirty -six 
0.  5 -inch -diameter  frosted  panel  lamps.  The  lamps  are  centered  behind  the  cut- 
uuL  areas  in  the  electroluminescent  panel.  The  individual  lamps  are  illuminated 
as  Hie  observer  presses  the  keys  of  a  30 -unit  push-button  response  panel  that 
is  external  to  the  apparatus.  These  lights  provide  feedback  information  to  the 
observer  concerning  his  response  to  indicate  the  position  of  the  change  he  detec¬ 
ted  in  the  plioLographie  stimulus.  The  feedback  is  immediate  upon  pressing  of  the 
response  key,  but  if  he  is  in  error,  the  circuitry  permits  him  to  release  the  key 
within  0.5  second  without  any  recording  of  that  response. 

At  the  eyepiece,  the  arrangement  of  optics  is  designed  to  produce  a  luminous 
cone  of  lighL  3.5  inches  in  diameter,  homogeneous  in  distribution  of  intensity 
and  color.  The  viewing  distance,  or  the  eye-lo-slide  distance,  is  21.5  inches. 
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Figure  2  FUNCTIONAL  DIAGRAM  CHANGE  DETECTION  APPARATUS 


Therefore,  a  0.  0062 -inch  object  in  the  photograph  subtends  a  visual  angle  of 
1.0  minute.  At  a.  photograph -to -ground  scale  ratio  of  1:14,500  as  used  in 
Experiments  B,  C,  and  D  described  below,  an  automobile  15  feet  long  would 
subtend  2  minutes  of  visual  angle. 

The  illuminance  level  used  in  the  initial  study  was  3.5  foot-candles  as  measured 
by  a  MaeBeth  illuminometer  at  the  eyepiece.  This  represents  the  illuminance 
of  the  individual  optical  paths  without  any  slides  in  the  holders.  While  this 
illuminance  level  was  adequale  for  the  simple  geometric  stimuli  of  the  firsL 
study,  it  had  lu  be  increased  later  for  aerial  photographic  slides  with  their 
greater  overall  density.  For  all  subsequent  experiments,  the  single  light 
source  was  replaced  by  twu  General  Electric  DEF  projection  lamps  ol  21  volts 
and  150  watts.  This  lamp  has  an  internal  parabolic  reflector  with  a  focal  point 
at  about  1.5  inches  in  front  of  the  lamp.  These  lamps  were  positioned  so  that 
the  conical  shutter  passed  through  the  point  of  convergence  of  the  rays,  and  the 
biconcave  lenses  were  removed  from  the  system  because  the  diverging  rays  be¬ 
yond  the  shutter  provided  a  sufficiently  large  illuminated  area  on  the  opal  screens. 
The  illuminance  level  was  raised  to  10  loot-candles  through  this  modification. 

The  response-panel  equipment  consisted  of  36  push  buttons  arranged  in  six 
columns  and  six  rows.  An  elevated  divider  separated  this  matrix  into  four 
quadrants  of  nine  push  buttons  each.  The  divider  was  just  higher  than  the  key 
height  and  served  as  a  tactual  reference  for  the  observer’s  hand  position  on 
the  response  panel.  Most  observers,  as  they  became  familiar  with  the  equip¬ 
ment  and  task,  used  this  tactual  reference  in  combination  with  the  visual  feed¬ 
back  provided  by  the  coding  matrix.  Connected  with  this  response  panel  was  a 
digital-to -analog  converter  that  provided  the  input  to  a  Hewlett-Packard  digital 
recorder  Model  560-A  printer.  The  readout  was  digital  in  form.  This  record 
provided  elapsed  time  from  the  start  of  a  trial  to  each  detection  of  a  change  as 
indicated  by  the  activation  of  a  response  key.  Associated  with  each  of  the  36 
response  keys  was  a  digital  representation  of  the  location  of  the  response.  On 
the  side  of  the  response  unit  was  a  thirty-seventh  key,  which  signaled  that  the 
oberver  had  completed  his  investigation:  with  its  activation,  the  printer  recorded 
a  summation  of  elapsed  time.  In  summary,  the  recording  equipment  provided 
these  dependent  measures: 

1)  Time  in  seconds  from  the  beginning  of  a  trial  to  each  detection  of  a  change: 

2)  Total  time  the  observer  used  in  the  trial: 

3)  The  location  of  the  identified  change  as  one  of  36  possible  outlined  areas 
within  the  4 -inch  by  4 -inch  display. 

The  four  experiments  immediately  following  were  preliminary  to  those  discussed 
in  later  sections  of  this  report  as  part  of  the  contract  work  specified  in  the 
Foreword. 
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Experiment  A 


The  first  experiment  investigated  the  efficiency  of  detecting  changes  in  syn¬ 
thetic  pictorial  data  as  a  function  of  presentation  method,  type  of  change,  and 
target  size,  A  3  by  3  by  3  factorial  design  was  used,  as  shown  in  Figure  3. 

The  three  presentation  methods  are  shown  on  the  abscissa;  the  ordinate  repre¬ 
sents  three  of  the  five  possible  ways  of  classifying  types  of  change,  according 
to  number,  size,  and  position  — the  variables  for  the  investigation.  The  syn¬ 
thetic  stimuli  were  high-contrast  photographs  of  squares  in  a  homogeneous  back¬ 
ground.  On  the  Z  axis,  the  sizes  of  these  squares  were;  Size  1,  4  minutes  of 
visual  angle;  Size  2,  16  minutes;  and  Size  3,  32  minutes.  The  matrix  includes 
27  individual  cells. 

Figure  4  further  defines  the  individual  makeup  of  one  of  these  cells.  Each  pair 
of  slides  was  presented  twice,  but  rotated  180  degrees  for  the  second  exposure. 
Slides  contained  either  zero,  two,  three,  or  six  changes;  this  number  was  unknown 
to  the  subjects. 

Fifteen  observers,  who  were  previously  trained  on  each  of  the  three  methods  of 
presentation,  reported  the  detected  changes  by  location,  with  response  time 
recorded  by  a  digital  printer.  An  average  of  8.  5  seconds  per  slide  was  needed 
to  report  the  average  of  three  changes. 

The  dependent  variables  for  these  studies  were:  (1)  average  inspection  per 
slide,  and  (2)  the  error  scores,  which  were  divided  into  two  categories; 
unreported  changes  and  false  reports  of  change. 

Experiments  B  and  C 

The  second  and  third  experiments  used  aerial  photography  with  two  levels  of 
"visual  noise."  The  stimulus  material  was  derived  from  36  photographs  ob¬ 
tained  from  the  Central  Intelligence  Agency.  The  "minimal  visual  noise" 
photographs  were  operationally  identical  photographs,  taken  from  the  Airplane 
Position  A  in  Figure  5,  with  target  changes  systematically  introduced  by  an 
artist  in  one  copy.  The  "high  visual  noise"  photographs  were  a  comparison  of 
Picture  A  incorporating  the  changes  introduced  by  the  artist  with  Picture  B 
of  the  same  area  taken  from  Position  B  of  Figure  5.  The  two  sets  of  photo¬ 
graphs  were  in  fact  obtained  on  the  same  pass  within  a  brief  interval.  The 
"high  visual  noise"  is  a  correlated  noise  in  that  it  is  a  product  of  the  two 
different  views  of  all  three-dimensional  ground  objects,  resulting  in  a  common 
effect  throughout  the  photograph. 

The  changes  introduced  on  the  2.  4 -power  enlargements  are  listed  in  Appendix  II. 
These  changes  are  distributed  among  four  classifications:  thirty  buildings, 
seven  transportation  vehicles,  seven  industrial  storage  areas,  and  nine  natural 
landscape  areas.  The  changes  were  located  within  the  6  by  6  sector  matrix  as 
shown  in  Figure  6.  With  reference  to  experimental  design  (Figure  7),  it  would 
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Figure  4  EXPERIMENT  A:  EXPERIMENTAL  DESIGN 
For  Individual  Cell 
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Figure  5  AERIAL  PHOTOGRAPHIC  CONDITIONS  REPRESENTATIVE 
OF  THE  COMPARATIVE  COVER  PHOTOGRAPHS  USED 
AS  THE  HIGH  VISUAL  NOISE  STIMULI  OF  EXPERIMENTS 
C  AND  D  (AND  CONTRACT  EXPERIMENT  I) 
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TRAINING  SLIDES 


Figure  6  FREQUENCY  OF  TARGET  CHANGE 
IN  VARIOUS  PARTS  OF  PICTURES 
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have  been  desirable  to  have  changes  occurring  with  equal  frequency  in  nil  36 
sectors  of  the  matrix.  It  also  would  have  been  desirable  to  have  these  sectors 
assigned  at  random  throughout  the  set  of  test  photographs.  However,  these  con¬ 
ditions  could  not  be  met  because  of  the  need  for  the  appearance  of  realism  in  the 
changes  introduced  in  the  photography. 

In  general,  the  original  negatives  were  of  high  image  quality  and  contrast. 
Fourth-generation  reproductions  of  the  original  scale  (1:14,496)  were  used  in 
the  comparator. 

The  average  integrated  transmission  value  for  the  36  standard  slides  was  22.53 
percent  and  the  transmission  varied  among  these  slides  to  produce  a  standard 
deviation  of  5.63  percent.  The  Position  B  members  of  the  pairs,  which  were 
always  compared  with  their  standard  counterparts,  had  a  mean  integrated 
transmission  of  19.48  percent  and  a  standard  deviation  of  5.12  percent.  With 
the  beam  intensity  set  at  10.  0  foot-candles,  the  average  illuminance  of  the  slides 
was  2.1  foot-candles,  and  the  average  brightness  difference  between  the  paired 
stimuli  was  0.30  foot-candle. 

The  individual  target  transmissions  and  the  average  immediate  surroundings 
were  measured  with  a  densitometer.  The  54  targets  were  distributed  about 
equally  as  to  negative  and  positive  contrast;  29  targets  were  darker  than  their 
surroundings.  The  percentage  contrasts  were  distributed  as  shown  in  Table  II. 


Table  II  DISTRIBUTION  OF  54  CHANGED  TARGETS 
BY  PERCENT  CONTRAST 


PERCENT  CONTRAST 

NUMBER  OF  TARGETS 

0  -  9.9 

2 

10  -  29.9 

17 

30  -  59.9 

18 

60  -  89.9 

11 

90  -  + 

6 

The  results  of  Experiments  A,  B,  and  C  indicate  that  an  observer  could  detect 
changes  faster  and  more  accurately  with  the  apparent-motion  presentation  method 
than  the  side-by-side  method.  This  was  true  for  simple  geometric  forms  as  well 
as  lor  complex,  "high  visual  noise”  photographs.  The  average  time  and  average 
omission  errors  (for  the  three  experiments)  are  shown  in  Figure  9.  Table  III 
gives  the  average  number  of  commission  errors  per  subject  per  picture. 
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Table  III  COMMISSION  ERRORS  FROM  DETECTION  OF 
CHANGE  EXPERIMENTS  A,  B,  AND  C 


METHOD 

SIDE  BY  SIDE 

OVERLAY 

APPARENT 

MOTION 

Simple  Geometric  Forms,  n  =  64 

0.1 

0.2 

0.1 

Photographs  With  Limited 

Visual  Noise 

0.1 

1.2 

0.5 

Photographs  With  High 

Visual  Noise 

0.4 

0.8  | 

■  -I 

0.7 

Values  =  Average  number  of  false  reports  per  picture  per  observer. 


The  overlay  method  yielded  overall  inspection  times  comparable  to  the  apparent- 
motion  method  and,  lor  geometric  stimuli,  a  comparable  proportion  of  current 
detections.  However,  the  overlay  method  yields  considerably  fewer  correct 
detections  than  the  apparent -motion  method  when  the  changes  sought  are  in 
aerial  photographs.  With  the  aerial  photographs,  the  overlay  method  also 
yields  lewer  correct  detections  than  the  side-by-side  method.  The  latter  is 
in  sharp  contrast  with  the  results  for  geometric  stimuli.  The  explanation 
might  rest  with  some  observed  differences.  The  bas-relief  and  Titchener's 
scintillation  e fleet  were  observable  with  the  geometric  stimuli  but  were  seldom 
ll  ever  seen  with  the  photographs.  Also,  a  high  frequency  of  meaningless  shapes 
was  observed  that  obscured  the  relevant  changes. 

Experiment  D 

The  lourth  experiment  on  efficiency  in  the  detection  of  change  compared  the 
periormance  ol  non-photointerprclers  and  photointerpreters  on  "high  visual 
noise"  serial  photographs  for  the  side-by-side  and  apparent-motion  methods 
of  presentation. 

Nine  photointerpreters  ol  the  United  States  Army  from  Fort  Lewis,  Washington, 
participated  as  observers.  The  nun -photointerpreters  were  nine  Hoeing  em¬ 
ployees  selected  only  as  having  adequate  visual  performance  at  21.5  inches  from 
the  eye.  Each  pholointerpreLer  was  paired  with  a  non -photo interpreter  by  random 
assignment,  and  each  pair  worked  with  the  same  photographs  under  the  same 
conditions. 
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DETECTION  TASK 

TIME,  OMISSION  AND  COMMISSION 
ERRORS 
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Figure  8  EXPERIMENT  D:  DESIGN 


2:5 


Figure  9  SUMMARY  BAR  GRAPH  OF  RESULTS  FROM  EXPERIMENTS  A, 


A  2  by  2  by  3  design  was  used  in  this  experiment  (Figure  8)  —  two  levels  of 
interpreter  experience,  two  display  methods,  and  three  types  of  changes. 

All  36  "high  visual  noise1'  photographs  were  used  although  any  one  oberver 
saw  only  24,  12  under  each  display  condition.  All  18  ohervers  made  observa¬ 
tions  under  both  display  conditions  as  well  as  all  types  and  numbers  of  changes. 

The  photointerpreter  mean  performance  on  the  detection  of  change  task  was 
not  statistically  significantly  faster  or  more  accurate  than  the  non -photo inter¬ 
preters  for  either  the  side-by-side  or  the  apparent-motion  methods  (Table  IV); 
seven  of  the  nine  non-photointerpreters  equaled  the  performance  of  the  photo¬ 
interpreters  on  the  detection  of  change  task.  This  suggests  the  hypothesis 
that,  in  the  detection  task,  semitrained  persons  are  comparable  to  photo- 
inter  prefers. 


Table  IV  MEAN  PERFORMANCE  OF  NON-P.I.  AND  P.l.  IN  DETECTING 
CHANGE  IN  COMPARATIVE-COVER  PHOTOGRAPHY 


SIDE-BY-SIDE 

APPARENT 

MOTION 

OMISSION 

OMISSION 

TOTAL  TIME 

ERRORS 

TOTAL  TIME 

ERRORS 

SECONDS 

(PERCENT) 

SECONDS 

(PERCENT) 

Photointerpreters 

m 

75,9* 

61.3* 

34.0* 

Non-Photointerpreters 

mam 

77.8 

.  63.8 

41.3 

*  No  significant  difference  between  photointerpreters  and  nan-photointerpreters  as 
indicated  by  Mann-Whitney  U  Test. 


Values  are  mean  per  subject  per  photograph. 


These  four  experimental  studies  yield  quantitative  estimates  of  the  relative 
effectiveness  of  the  standard  side-by-side  method  lor  detecting  change  in  com¬ 
parative-cover  photography.  These  data  were  gathered  under  conditions 
similar  to  the  operational  method  of  comparing  aerial  photographs.  Opera¬ 
tional  techniques  differ  principally  in  the  degrees  of  freedom  available  to  the 
interpreter  (i.  e. ,  group  discussions,  varying  the  viewing  distance,  and  physical 
measurements  on  the  transparency  or  print).  The  influences  of  these  freedoms 
are  not  quantitatively  known,  but  they  would  undoubtedly  limit  generalizations 
from  the  present  data. 


Because  the  task  oi'  detecting  changes  should  have  been  relatively  easy  (the 
only  differences  between  pictures  were  the  target  changes  to  bo  reported, 
with  no  irrelevant  "noise'*),  the  high  percentage  of  omission  errors  (fill  percent) 
with  the  side-by-side  method  is  disturbing.  That  is.  according  to  the  appar¬ 
ently  reasonable  assumption  that  the  operational  task  is  yet  more  difficult,  the 
error  rate  is  surprisingly  high*  If,  as  it  appears,  our  estimate  of  86  percent 
omission  errors  is  conservative,  efforts  to  reduce  this  error  would  be  weli 
spent.  The  conservativeness  of  this  estimate  is  supported  by  a  10-percent 
increase  in  omission  errors  as  a  function  of  the  introduction  of  "noise"  asso¬ 
ciated  with  a  difference  in  camera  angle  of  58  degrees*  Additional  errors 
would  be  anticipated  with  differences  between  images  related  to  atmospheric 
effects,  seasonal  changes,  film-processing  changes,  and  equipment 
malfunctions. 

SUMMARY  OF  RESULTS  AND  CONCLUSIONS 

The  quantitative  information  provided  by  these  four  experimental  studies  indi¬ 
cates  that; 

1)  The  effectiveness  of  the  operational  or  customary  side-by-side  method  of 
displaying  photographs  for  the  detection  of  change  can  be  measured,  and 
it  can  be  compared  with  other  display  methods*  The  method  of  preparing 
stimuli  by  introducing  controlled  changes  in  aerial  photographs,  or  the  use 
of  synthetic  stimuli,  provides  a  zero  reference  or  a  base  against  which  to 
evaluate  interpreter  responses.  Such  a  base  or  "ground  truth"  is  seldom, 
if  ever,  found  in  operational  reconnaissance,  especially  in  time  of  war. 

To  have  an  absolute  criterion  makes  it  possible  to  have  a  measure  of  com¬ 
pleteness  or  a  proportion  of  the  changes  detected  to  the  changes  introduced. 
This  completeness  score  was  60  percent  for  synthetic  stimuli  of  low  com¬ 
plexity,  33  percent  for  "low  noise"  aerial  photographs,  and  24  percent  for 
the  "high  noise"  photographs.  The  latter  score  of  24  percent  was  the  com 
pleloness  score  found  for  both  the  non -photointerp role1  rs  and  the  photo- 
interpreters. 

2)  The  display  method  can  significantly  affect  manfs  performance  in  detecting 
change.  In  all  four  experiments,  a  significant  improvement  in  inspection 
time  and  performance  completeness  was  demonstrated  with  the  apparent- 
motion  display  method.  With  only  a  single  exception  (overlay  method  versus 
apparent  moliun  for  synthetic  stimuli  of  low  complexity),  the  apparent- motion 
method  proved  to  be  significantly  faster  than  the  other  display7  methods:  in 
every  comparison,  the  detection  of  change  reports  were  more  complete. 

The  apparent-motion  method  proved  better  than  the  side-by-side  method  in 
completeness  (!)i)  percent  versus  68  percent)  when  the  stimuli  were  of  low 
complexity.  This  difference  increased  slightly7  as  stimulus  complexity7  and 
"visual  noise"  increased. 


3)  The  experience  level  of  the  observer  was  not  a  significant  factor  in  the 
task  of  detecting  change  in  comparative-cover  photography.  In  the  detec¬ 
tion  of  change  task,  the  professional  training  and  experience  of  U.S.  Army 
photointerpreters  did  not  significantly  help  them  in  detecting  changes  in 
aerial  photographs  with  either  the  side-by-side  or  apparent-motion  display 
methods  when  their  performance  was  compared  with  observers  trained 
only  to  use  the  specific  displays. 

4)  The  positive-negative  overlay  method  was  not  an  improvement  over  the 
standard  method  of  displaying  aerial  photographs  for  the  detection  of 
changes.  This  method  —  one  that  appears  most  applicable  to  machine 
usage  —  is  not  an  effective  method  of  displaying  for  man's  discrimination 
the  comparative-cover  photographs  of  high  complexity  and  of  many  gray 
steps.  When  the  photographic  stimuli  are  regular  configurations  of  low 
complexity  and  few  gray  steps,  the  overlay  method  is  a  significant  improve¬ 
ment  over  the  side-by-side  display  method.  The  explanation  offered  for 
this  difference  in  performance  with  simple  and  complex  stimuli  is  that  the 
perceptual  phenomena  that  assist  in  the  low -complexity  discrimination  are 
not  available  to  the  observer  when  working  with  stimuli  of  high  complexity. 
The  ’'silvery"  appearance  of  Titchener's  scintillation  effect,  and  the  bas- 
relief  effect,  both  associated  with  unchanged  areas,  were  ineffective  with 
the  complex  stimuli.  In  addition,  changes  are  masked  by  the  positive¬ 
negative  interaction  that  produces  unusual  and  meaningless  shapes. 

5)  A  number  of  conditions  are  related  to  man's  performance  in  detecting 
changes  in  aerial  photographs.  In  low-complexity  stimuli,  changes  in  the 
10-minute  (visual  angle)  and  32 -minute -si zed  squares  were  found  signifi¬ 
cantly  faster  than  the  same  kinds  of  changes  introduced  in  the  4-minute¬ 
sized  objects.  (The  performance  in  detecting  change  did  not  differ  signifi¬ 
cantly  for  the  16-minute  objects  when  compared  with  the  32-minute  objects.) 
In  these  instances,  the  changes  wore  half  the  length  of  the  side  of  the  square 
targets.  For  the  low -complexity  stimuli,  the  types  of  changes  in  their  order 
of  difficulty  were  position,  size,  and  addition.  No  significant  differences 
were  found  among  these  types  of  changes  when  they  occurred  in  the  aerial 
photographs.  Orientation  of  the  slide  did  not  significantly  change  perform¬ 
ance.  Each  of  the  differences  mentioned  above  was  found  to  be  significant 

at  or  below  the  0.01  probability  level. 

The  value  of  stimulus  parameters  that  were  held  constant  in  this  former  work 
are  given  below  for  reference.  The  rate  of  alternation  in  the  apparent-motion 
display  was  always  1.5  cps,  where  one  cycle  is  the  successive  presentation  of 
both  members  of  the  pair.  The  photograph  as  presented  to  the  observer  was 
always  a  4-inch  by  4 -inch  square.  The  aerial  photographic  display  scale  was 
1:14,490,  and  only  transparencies  were  used.  The  changes  in  the  photographs 
were  all  within  16  to  32  minutes  of  visual  angle.  The  original  quality  of  the 
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aerial  photographs  was  excellent,  and  the  observers  saw  fourth -generation 
reproductions  of  these  photographs.  No  special  treatment  of  the  stimuli  was 
made  in  changing  and  matching  along  the  dimensions  of  magnification  or  recti¬ 
fication.  Registration  in  X  and  Y  coordinates  was  as  good  as  could  be  accom¬ 
plished  with  hand  mounting  and  visual  matching.  The  magnitude  of  registration 
errors  was  later  measured,  and  was  as  shown  in  Table  V. 

Table  V  REGISTRATION  ERRORS  MADE  IN  MOUNTING  THE  144 
TRANSPARENCIES,  FOUR  FOR  EACH  OF  36  PAIRS 
Used  As  Stimuli  In  Experiments  A,  B,  And  C 


REGISTRATION  ERRORS 

IN  MILLIMETERS 

WHEN 

MATCHED  TO  MASTER  OF  EACH  PAIR  | 

Mean  in  X 

SD  in  X 

Mean  in  Y 

SD  in  Y 

Negatives 

-0.029 

mm 

+0.044 

0. )  o2 

Positives  (Low  Noise) 

-0.031 

■SH 

0.161 

Positives  (High  Noise) 

+0.079 

0.335 

1 

0.361 

OBJECTIVES  OF  THE  RESEARCH  PROGRAM 


The  importance  of  the  reconnaissance  system  as  a  primary  source  of  information 
has  been  discussed  in  the  initial  pages  of  this  report.  The  speed  with  which  this 
information  system  must  react  has  been  gaining  emphasis  with  each  new  ballistic 
weapon  development.  The  potential  now  exists  for  any  power  to  inflict  major 
destruction  on  an  adversary  within  three  quarters  of  an  hour  from  its  first  war- 
like  act.  Reconnaissance  system  reaction  time  must  be  faster  than  this  strike 
time.  This  intensification  of  the  requirement  for  speed  has  brought  into  even 
sharper  contrast  the  performance  of  the  interpretation,  classification,  and  en¬ 
coding  subsystems  with  those  of  collection,  storage,  and  retrieval. 

This  need  for  speed  and  accuracy  has  re-emphasized  the  magnitude  and  the 
immediacy  of  the  need  to  increase  the  flow  of  information  through  the  recon¬ 
naissance-intelligence  system. 

The  research  reviewed  in  the  preceding  pages  was  directed  at  this  specific 
problem  area,  but  has  not  provided  sufficient  definitive  data  on  the  limited  sub¬ 
ject  of  the  human  observer’s  performance  in  detecting  changes  in  comparative- 
cover  photographs.  The  U.S.  Air  Force  has  pointed  out  broad  areas  that  have 
not  been  investigated.  They  indicated  that  research  data  did  not  exist  to  permit 
an  adequate  prediction  of  the  photointerpreter’s  performance  with  different 
qualities  of  photographs,  for  different  parts  of  the  reconnaissance  job,  with 
current  or  new  equipment,  or  whether  current  training  procedures  were 
adequate. 

This  research  program,  then,  was  designed  to  provide  new  data  on  photo¬ 
interpreter  performance  as  a  function  of  those  variables  that  appeared  most 
important  and  susceptible  to  quantitative  manipulation.  The  objective  of  the 
program  was  to  determine  the  effects  of  selected  variables  on  the  performance 
of  photointerpreters  in  accomplishing  the  tasks  of  screening  and  identifying 
target  changes  when  the  side-by-side  and  apparent-motion  display  methods  are 
employed  in  presenting  comparative-cover  aerial  photographs. 

The  variables  selected  for  investigation  include: 

1)  Display  method; 

2)  Photointerpreter  experience  level; 

3)  Contrast; 

4)  Image  quality; 

5)  Photographic  scale: 

6)  Terrain  complexity; 
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7)  Relevancy  of  changes: 

8)  Time  Lapse  between  photographs; 

9)  Types  of  change: 

10)  Real  versus  artificial  changes. 

The  procedural  steps  in  implementing  this  program  were:  (1)  to  obtain,  with 
the  assistance  of  the  Air  Force,  samples  of  comparative-cover  photography 
that  were  representative  of  military  and  civil  x'econnaissance  pictorial  data, 

(2)  to  conduct  three  major  experiments  to  determine  the  effects  of  the  above 
variables  on  photointorpreter  performance  with  the  two  display  methods  in  the 
tasks  of  screening  and  identifying  change,  and  (3)  to  evaluate  the  significance 
of  the  experimental  data  with  respect  to  the  relative  advantages  and  disadvan¬ 
tages  of  the  two  methods  of  display  as  well  as  provide  recommendations  for 
future  investigations.  Common  to  the  three  experiments  are  the  primary 
variables  of  display  method  and  photointerpreter  experience  level.  These 
variables  and  the  other  selected  variables  were  apportioned  as  shown  in  Table 
VI. 

Table  VI  SELECTED  VARIABLES  APPORTIONED  BY  EXPERIMENTS 


EXPERIMENT  I 


INTRODUCTION 

The  experiments  described  in  this  section  and  those  that  follow  are  a  direct 
extension  of  a  continuing  program  of  a  reconnaissance  study  program  conducted 
in  the  Engineering  Psychology  Laboratory  at  The  Boeing  Company.  The  goal  of 
this  research  program  is  to  provide  information  applicable  to  the  design  and 
evaluation  of  display  systems  for  faster,  more  accurate,  and  more  complete 
information  extraction  from  comparative -cover  aerial  photography  than  present 
systems  provide. 

In  addition  to  greater  speed  and  accuracy,  a  better  display  will  require  less 
training*  for  efficient  observer  performance.  Considerable  economy  might  be 
realized  if  inexperienced  observers  are  able  to  detect  changes  and  at  least 
roughly  classify  them,  allowing  photo  inter  prefers  to  spend  more  time  on  more 
complex  interpretation.  The  only  way  to  evaluate  the  inexperienced  observer 
in  this  situation  is  to  compare  his  performance  with  experimental  displays  to 
the  performance  of  trained  photo  interpreters  with  the  same  equipment.  Photo¬ 
interpreters  and  non -photointerpreters  were  used  in  each  experiment. 

The  quality  of  the  imagery  received  by  the  photointerpreter  in  operational  situ¬ 
ations  varies  widely  from  sample  to  sample.  One  such  dimension  of  quality  is 
film  contrast.  Film  contrast  is  distinguished  from  inherent  target  contrast, 
which  refers  to  the  differences  in  film  density  arising*  from  reflectance  differ¬ 
ences  of  the  target  object  and  its  immediate  surroundings.  Film  contrast  in 
any  sample  photograph  depends  variously  on  such  things  as  the  sensitivity  of 
the  emulsion,  and  the  development  and  reproduction  procedures. 

Experiment  I  investigated  the  effects  of  three  levels  of  contrast  on  the  detection 
and  identification  of  change.  Through  the  systematic  introduction  into  the  photo¬ 
graphy  of  changes  in  target  size,  position,  and  number  (additions  or  subtractions), 
"absolute  ground  truth"  was  established  for  the  test  imagery.  This  made  is  pos¬ 
sible  to  determine  empirically  the  probabilities  of  detecting  and  identifying  each 
of  these  types  ol  change  as  affected  by  various  combi  nations  of  viewing  condition, 
image  quality,  and  observer  experience. 

Briefly  summarized,  the  objectives  of  Experiment  I  were  to  determine  the 
accuracy,  completeness,  and  speed  with  which  photo  interpreters  and  non¬ 
photo  interpreters  can  detect  and  identify  changes  of  number,  size,  and  position 
occurring  in  comparative -cover  aerial  photography  of  three  contrast  levels 
when  these  photographs  are  displayed  in  side-by-side  and  apparent-motion  formats. 


31 


SUBJECTS 


Although  the  goals  of  the  study  involved  generalization  of  the  results  to  Air 
Force  photointerpreters,  no  feasible  approach  for  sampling  this  population 
could  be  found,  A  compromise  was  made.  Through  a  request  placed  in  the 
weekly  Boeing  newspaper,  GO  Boeing  employees  were  found  who  had  received 
training  and/or  experience  in  photointerpretation,  although  they  were  not  cur- 
rently  engaged  in  it.  A  brief  summary  of  the  training  and  experience  of  those 
who  served  on  a  volunteer  basis  in  this  research  program  is  contained  in 
Appendix  I.  It  can  be  seen  here  that,  although  12  out  of  the  21  persons  in  this 
group  had  received  their  training  and  gained  their  experience  in  a  military 
setting,  training  of  others  was  obtained  as  part  of  an  academic  program  and 
whose  experience  with  aerial  photography  was  gained  in  the  course  of  such 
duties  as  cartographers,  pliotogrammetrists,  foresters,  geologists,  and  sur¬ 
veyors.  Although  the  amount  and  specific  orientation  of  the  training  and  experi¬ 
ence  represented  in  this  group  is  quite  heterogeneous,  they  all  possess  one 
common  element;  they  are  all  familiar  with  the  use  of  aerial  photographs  for 
the  detection,  localization,  and  identification  of  ground  objects.  The  members 
of  this  subject  sample  were  designated  the  l,photointerpretersM  with  the  expec¬ 
tation  that  their  performance  under  the  different  experimental  treatments  would 
provide  some  approximation  of  what  might  be  expected  from  a  sample  of  cur¬ 
rently  active  military  photo  interpreters.  Twelve  of  these  subjects  constituted 
the  photo  interpreters  in  Experiment  I. 

A  second  group  of  12  observers  (the  "non -photo interpreters'1)  was  selected  from 
the  Bioastronautics  organization  of  The  Boeing  Company.  Approximately  equal 
proportions  of  shop  technicians,  laboratory  technicians,  and  junior  and  senior 
research  personnel  were  represented.  Only  those  persons  who:  (1)  had  no 
training  or  experience  in  photo  interpretation,  (2)  had  not  participated  in  any  of 
the  previous  experiments  in  the  reconnaissance  program  conducted  by  the  Engi¬ 
neering  Psychology  Unit,  and  (3)  did  not  demonstrate  any  limiting  visual  defects 
or  weaknesses  were  considered  for  inclusion  in  this  group. 

TRAINING 

Prior  to  the  beginning  oi  the  first  experimental  session,  each  subject  was  trained 
on  the  response  and  reporting  procedures  for  both  display  methods.  The  training 
program  consisted  of  presenting  each  subject  with  a  series  of  training  slides  that 
were  graded  in  difficulty. 

J he  iirst  set  ul  six  pairs  oi  training  slides  each  contained  G4  randomly  positioned 
white  squares  against  a  black  background.  Within  this  set  of  slides,  three  types 
ol  change  (number,  size,  and  position)  were  equally  represented  and  the  number 
ol  changes  per  slide  pair  varied  from  zero  to  six. 

The  second  set  of  training  materials  consisted  of  six  pairs  of  comparative-cover 
aerial  photographs  that  were  described  as  "minimum  visual  noise”  photographs  in 
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a  preceding  section  of  this  report.  These  comparative  pairs  were  simply  two 
prints  from  the  same  negative  with  only  a  small  number  of  target  objects  artis¬ 
tically  added  or  changed.  Changes  again  represented  all  three  classes  and 
varied  in  number  per  pair  to  prevent  the  observer  from  developing  a  response  set. 

The  final  set  of  training  slides  in  the  series  consisted  of  six  samples  of 
comparative -cover  aerial  photography  taken  on  the  same  pass  and  represented 
a  viewing  angle  separation  of  approximately  58  degrees.  As  in  the  above- 
mentioned  slides,  each  of  the  three  types  of  change  was  represented,  and  the 
number  of  changes  per  comparative  pair  was  varied. 

The  instructions  (see  ’‘Instructions'1  below)  were  to  detect  and  report  all  changes 
existing  between  the  two  photos  of  each  comparative  pair.  Knowledge  of  results 
and  instructional  comments  were  provided  by  the  experimenter  at  each  step  of 
the  training  procedure.  All  questions  were  answered  directly  by  the  experi¬ 
menter  if  the  answer  would  not  bias  or  invalidate  the  subject's  performance 
during  the  remainder  of  the  testing. 

Previous  studies  had  demonstrated  that  this  training  program  was  adequate  to 
minimize  both  target  localization  error  and  response  time  attributable  to  lack 
of  familiarity  with  the  equipment. 

INSTRUCTIONS 

The  following  instructions  were  presented  during  the  training  session. 

"Please  give  your  full  attention  to  the  following  instructions:  The  primary 
purpose  of  this  investigation  is  to  evaluate  the  relative  merits  of  two  dif¬ 
ferent  display  techniques. 

"The  material  to  be  displayed  is  comparative -cover  aerial  photography  from 
which  inteiligcnce  information  is  to  be  extracted.  Comparative -cover  photo¬ 
graphy  consists  of  two  pictures  of  the  same  area  taken  at  two  different  times. 
The  'intelligence  information'  that  we  are  asking  you  to  look  for  is  repre¬ 
sented  by  the  changes  which  have  occurred  between  the  time  of  the  first  and 
the  second  photograph. 

"The  two  display  techniques  being  evaluated  are  referred  to  as  the  •side-by- 
side 1  method,  in  which  the  two  pictures  are  presented  simultaneously  in 
adjacent  positions,  and  the  'apparent-motion'  method,  in  which  the  two  pic¬ 
tures  are  rapidly  alternated  in  the  same  location  so  that  those  objects  which 
are  the  same  in  both  pictures  remain  relatively  stationary  while  those 
objects  which  have  changed  appear  animated  or  in  motion. 

"You  have  been  asked  to  serve  as  an  observer  or  photo  interpreter  in  this 
research  program  to  provide  the  performance  data  which  will  allow  us  to 
make  the  desired  evaluation  of  these  two  display  methods.  As  an  observer 
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in  this  first  experiment  you  will  be  asked  to  serve  on  tour  separate  occa¬ 
sions,  each  lasting  approximately  one  hour.  During  this  first  session  you 
will  be  given  training  on  both  of  the  display  techniques  and  will  be  taught  to 
use  the  response  system  which  wc  have  designed  to  obtain  our  performance 
measures.  In  the  remaining  sessions  you  will  be  asked  to  view  a  number 
cf  aerial  photographs  under  each  of  the  display  methods  and  to  report  the 
location  of  changes  which  have  occurred  between  the  two  photographs  in 
each  comparative  pair, 

"We  will  now  begin  by  teaching  you  to  use  the  response  system.  First,  take 
this  shutter-activation  switch  in  your  left  hand.  By  pressing  the  button  on 
the  top  of  this  handle  you  can  open  the  shutter  in  the  display  apparatus 
thereby  letting  you  view  the  photographs  displayed  inside.  During  the  re¬ 
mainder  of  this  training  session  and  during  the  subsequent  testing,  any 
time  you  wish  to  look  at  the  photographs  you  simply  press  this  shutter  switch. 
Although  we  do  not  have  any  pictures  displayed  in  the  box  now,  you  can  try 
activating  the  shutter  a  few  times  to  see  how  this  component  operates. 

"Now  if  you  will  open  the  shutter  again  we  will  continue  on  to  the  next  step. 

By  looking  through  the  viewing  aperture  you  will  be  able  to  see  a  4"  x  4" 
square  of  light.  Later  in  the  training,  aerial  photographs  will  be  seen  in 
this  area  during  the  ’apparent-motion'  presentation.  With  the  ’side-by- 
side'  presentation  two  such  areas,  one  slightly  to  the  left  of  center  and  a 
second  slightly  to  the  right  of  center,  will  be  seen. 

"The  response  panel,  which  you  will  use  to  indicate  the  areas  in  which  you 
detect  a  change,  is  located  here  on  your  right.  You  will  notice  that  the  .36 
pushbuttons  located  on  the  response  panel  are  arranged  in  a  square;  six 
rows  and  six  columns.  Each  of  these  pushbuttons  corresponds  to  a  differ¬ 
ent  portion  of  the  photograph.  When  any  one  of  these  buttons  is  depressed 
a  small  light  appears  in  the  corresponding  position  over  the  photograph. 

For  instance,  pressing  the  button  in  the  upper  left  of  the  response  panel 
causes  a  light  to  appear  in  the  upper  left  corner  of  the  photograph  (or,  since 
there  is  no  photograph  in  the  system  now,  over  the  light  square). 

"Press  a  few  of  the  buttons  now  to  see  how  this  feedback  system  works,  but 
do  not  hold  the  buttons  down  for  more  than  a  half  second.  If  you  hold  a 
button  down  for  over  one-half  second  the  small  single  light  disappears  and 
a  ’light  veil’  comes  on  over  the  whole  area  except  over  the  area  in  which 
the  single  light  originally  appeared  and  the  three  immediately  adjacent 
sections.  This  will  permit  you  to  continue  looking  at  only  this  area  without 
being  distracted  by  other  things  in  the  picture. 

"Try  it  a  few  times  now.  You  will  notice  that  when  you  release  the  button  the 
shutter  closes  over  the  viewing  aperture.  You  can  open  the  shutter  again  by 
depressing  the  shutter  activation  switch  with  your  left  hand. 
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"The  one-half  second  delay  between  the  pressing  of  the  button  and  the  illumi- 
nation  of  the  veiling  mask  is  included  to  allow  you  ample  opportunity  to 
change  your  response  as  many  times  as  necessary  until  you  have  pressed 
the  right  button:  that  is.  the  button  which  corresponds  to  the  area  on  the 
photograph  which  contains  the  change  you  have  detected  and  wish  to  report. 

"Now  to  give  you  some  experience  in  using  the  response  panel  and  to  intro¬ 
duce  you  to  what  different  types  of  changes  look  like  with  the  two  different 
display  techniques  *  I  will  show  you  some  pictures  containing  a  collection  of 
white  squares  on  a  black  background.  Represented  on  these  pictures  arc 
three  kinds  of  change:  changes  in  size,  where  a  square  has  been  made 
Larger  or  smaller;  changes  in  number,  where  a  square  has  been  added  or 
removed;  and  changes  in  position,  where  a  square  has  been  shifted  (either 
vertically  or  horizontally)  or  rotated. 

"It  is  your  task  to  search  these  pictures  for  the  differences  that  exist  between 
the  two  photographs.  As  soon  as  you  detect  a  change,  either  in  size,  posi¬ 
tion,  or  number,  press  the  button  which  illuminates  the  light  over  that  area 
on  the  picture.  Hold  the  button  down  and  verbally  report  the  kind  of  change 
which  is  represented.  Then  release  the  response  button.  When  you  are 
ready  to  continue  the  search  open  the  shutter  with  the  left  hand  switch  and 
continue  the  search.  Repeat  this  procedure  until  you  are  satisfied  that  you 
have  detected  each  and  every  change  which  exists  on  the  photographs.  When 
you  are  sure  that  you  have  reported  all  the  changes,  press  the  small  red 
button  on  the  right  side  of  the  response  panel.  This  button  signals  the  com¬ 
pletion  of  search  for  that  picture  pair.  The  experimenter  will  then  tell  you 
how  well  you  have  performed  and  will  insert  a  now  sot  of  photographs." 

After  this  series  of  photographs  was  completed,  the  following  instructions  were 

given. 

"The  next  series  of  photographs  you  will  be  shown  are  aerial  photographs 
taken  between  Baltimore  and  Washington,  D.  C.  These  particular  photo¬ 
graphs  are  in  a  class  which  we  call  Tow  noise*  since  both  pictures  were 
taken  from  the  same  point  in  space.  Furthermore,  the  second  picture  was 
taken  very  shortly  after  the  first  was  taken  so  that  they  are  very  similar  in 
every  respect  (such  as  shadows). 

"In  viewing  these  photographs  you  will  report  the  location  of  the  changes  you 
detect  in  the  same  way  that  you  did  with  the  preceding  material,  that  is,  by 
pushing  the  button  which  corresponds  to  the  area  of  change  in  the  photographs. 
You  are  also  required  to  tell  the  type  of  change  which  you  have  found.  In 
addition  to  the  location  and  type  ol  change  you  will  now  be  requested  to  iden¬ 
tify  the  object  which  has  changed.  It  may  be  a  building  that  has  been  enlarged 
or  an  oil  tank  which  has  been  added  or  a  ship  which  has  moved  from  a  dock. 

Be  sure  you  have  found  all  the  changes  before  you  signal  for  the  next  sot  of 
pictures. " 
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After  this  series  of  photographs  was  completed,  the  following  instructions  were 
given. 

"This  last  set  of  pictures  is  very  similar  to  those  which  you  have  just  been 
looking  at.  The  main  difference  is  that  whereas  the  previous  pictures  were 
taken  from  the  same  point  in  space,  these  pictures  were  taken  from  slightly 
different  places.  It  is  as  if  the  airplane  taking  the  picture  was  not  able  to 
fly  exactly  the  same  flight  path  on  successive  missions  and  so  the  object  on 
the  ground  is  photographed  from  slightly  different  angles.  Although  all 
objects  on  the  ground  will  look  a  little  different  on  each  of  the  pictures  of  a 
pair  only  those  objects  which  have  definitely  been  changed  are  to  be  reported. 
The  reporting  procedure  will  be  the  same  as  that  which  you  used  with  the 
'low  noise!  material." 

Before  the  first  lest  session  was  begun,  a  brief  review'  of  the  response  system 
was  given,  and  the  following  instructions  were  presented. 

"Before  beginning  it  is  important  that  you  understand  how  your  performance 
is  being  evaluated.  We  are  interested  in  three  aspects  of  your  performance: 
the  SPEED  with  which  you  can  detect  the  changes;  your  ACCURACY  in  re¬ 
porting  the  location,  kind  of  change  and  object  changed;  and  how  many  real 
changes  you  can  detect.  We  are  equally  interested  in  all  three  measures: 
SPEED,  ACCURACY  and  COMPLETENESS.  Since  there  may  be  some  pic¬ 
ture  pairs  which  contain  no  changes  and  others  which  contain  several  changes 
you  are  cautioned  to  be  relatively  certain  of  your  decisions  before  you  re¬ 
spond.  To  call  something  a  change  when  it  is  not  really  changed  is  as  bad 
as  not  detecting  a  change  which  ks  present. 

"We  are  now  ready  to  begin.  Do  you  have  any  questions?" 

DESIGN 

The  independent  variables  in  this  experiment  were:  (1)  presentation  method  — 
side  by  side  and  apparent  motion,  (2)  experience  level  of  interpreter  —  inexperi¬ 
enced  and  experienced,  (3)  relative  film  contrast  —  20,  40,  and  GO  percent,  and 
(4)  type  of  target  change  —  number,  si/,e,  and  position.  A  schematic  represen¬ 
tation  of  the  treatments  and  treatment  levels  is  shown  in  Figure  10.  The  depen¬ 
dent  measures  were:  (1)  inspection  time,  (2)  number  of  correct  detections  and 
identifications,  and  (3)  errors  of  commission. 

The  design  chosen  for  measuring  differences  between  the  respective  levels  of 
contrast  and  the  two  display  methods  was  one  in  which  all  twelve  observers  in 
each  group  (experienced  and  inexperienced)  received  all  six  combinations  of 
experimental  treatments.  All  six  permutations  of  the  three  contrast  levels 
were  used  twice  (once  side  by  side  and  once  in  apparent  motion)  for  each  group 
of  subjects. 
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Each  subject  viewed  each  of  3G  comparative  pairs.  He  inspected  12  scenes  in 
each  of  three  experimental  sessions.  Six  of  the  scenes  presented  in  each  session 
were  displayed  in  side-by-side  format  and  six  in  apparent  motion.  Each  of  the 
three  types  of  target  changes  was  represented  in  the  comparative  pairs  seen 
during  each  session. 

Each  session  required  between  40  and  70  minutes,  a  length  that  appeared  to  be 
most  efficient  without  tiring  the  observer. 

The  analysis  was  conducted  as  if  the  design  were  a  completely  replicated  (12 
times)  factorial  arrangement.  Because  of  the  repeated  measures  on  the  same 
subjects,  it  was  anticipated  that  the  statistical  significance  of  obtained  treat¬ 
ment  differences  might  be  somewhat  enhanced.  However,  the  careful  balancing 
of  the  treatment  assignments  was  expected  to  minimize  this  effect. 

EQUIPMENT 

The  display  equipment  was  the  same  as  that  shown  in  Figure  2,  but  with  two 
modifications. 

The  first  modification  was  made  to  introduce  the  three  levels  of  contrast  reduc¬ 
tion,  A  new  two-way  mirror  was  installed  that  had  a  good  color  balance  and 
provided  30-percent  transmission,  30-percent  absorption,  and  30-percent  reflec¬ 
tance.  Contrast  was  varied  through  the  addition  of  a  fourth  optical  pathway 
(Figure  11).  A  plate-glass  surface  just  in  front  of  the  eyepiece,  and  inclined 
45  degrees  to  the  line  cf  sight,  reflected  the  light  coming  from  a  diffuse  "cold 
light"  source.  Neutral-density  filters  were  placed  between  the  light  source  and 
the  reflecting  plate  glass  to  change  the  intensity  of  this  glare  source.  Compen¬ 
sating  filters  were  introduced  just  after  the  conical  shutter  (Figure  11)  to  main¬ 
tain  a  constant  illuminance  level  at  the  eye. 

The  100 -percent  contrast  condition  was  10  foot -candles  of  illuminance  for  each 
major  beam  without  the  slide  in  its  holder.  The  60-,  40-,  and  20-  percent  con¬ 
trast  conditions  were  effected  by  the  substitution  of  4,  6,  and  8  foot -candles  of 
glare-source  illuminance  for  the  same  quantities  of  beam  illuminance.  The 
measurements  of  illuminance  were  made  at  the  eyepiece  with  a  Mac  Beth 
illumine  meter . 

The  second  modification  of  the  equipment  involved  a  redesigned  response- 
feedback  channel  to  obtain  independent  measures  of  detection  and  identification 
time.  In  the  description  of  this  feedback  channel  (Page  11),  the  individual- 
response  pushbuttons  for  indicating  target  location  (those  in  the  6  by  6  matrix) 
were  in  a  normally  open  circuit  with  a  corresponding  C  by  G  matrix  of  lights. 

As  the  observei  indicated  the  location  of  a  change,  one  of  these  lights  was  turned 
on  and  became  visible  to  the  observer  as  a  glare  veiling  that  one  thirty -sixth  of 
the  slide  corresponding  to  his  response.  This  same  feedback  system  was  main¬ 
tained  in  the  modification  of  the  equipment  with  the  exception  of  an  increased 
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EYEPIECE 

Figure  11  FUNCTIONAL  DIAGRAM  OF  MODIFIED 
CHANGE-DETECTION  APPARATUS 


intensity  and  u  diffusion  square  that  was  introduced  in  front  of  the  lamp.  This 
latter  changed  the  shape  of  the  veiling  light  from  a  circle  to  a  square  about 
0.G67  inch  on  a  side.  The  feedback  of  a  detection  response  looked  like  that  in 
Figure  12  for  the  first  0.5  second  the  button  was  depressed. 

The  printer  circuit  records  the  elapsed  time  from  the  beginning  of  the  exposure 
to  the  detection  response.  With  the  exception  of  the  square  image  of  the  veiling 
patch  for  detection  response  feedback,  this  response  and  readout  is  the  same 
as  in  previous  experiments  where  the  only  task  was  detection. 

The  method  of  separating  the  identification  task  from  the  possibility  of  continued 
search  during  the  time  the  verbal  response  was  being  given  was  to  obscure  all 
the  remainder  of  the  slide  except  the  designated  area  and  three  immediately 
adjacent  areas  which  provided  contextual  information  to  aid  in  identification. 
Figure  13  illustrates  this  masking  by  the  occluding  veil.  The  keying  circuit 
selected  and  switched  on  the  32  masking  lamps  0.  5  second  after  the  detection 
response  if  the  key  was  maintained  in  the  depressed  position  by  the  observer. 
The  observer  could,  as  before,  change  his  response  within  this  0. 5-second 
delay  interval  without  the  printer  recording  the  nonintended  response. 

During  his  reporting  of  the  identity  and  typo  of  change,  the  observer  maintained 
the  depressed  position  of  the  response  key,  Release  of  this  key  activated  the 
printer  to  record  the  termination  time  of  the  identification  response.  The  ob¬ 
server  repeated  the  sequence  of  operations  by:  (1)  opening  the  shutter  to  begin 
the  trial,  (2)  detecting  a  change,  (3)  depressing  the  response  key,  (4)  holding 
the  key  during  identification,  and  (5)  releasing  the  key,  until  he  had  responded 
to  all  changes  seen  in  the  photographs.  The  observer  indicated  that  he  had  com¬ 
pleted  his  inspection  by  pushing  a  final  button  that  recorded  total  inspection  time 
on  the  digital  printer. 

STIMULUS  MATERIALS 

The  stimulus  materials  were  the  same  as  those  in  Experiments  G  and  D 
reported  in  the  "Introduction,  "  as  well  as  the  detailed  information  in  Appendix  II. 
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Figure  12  FEEDBACK  IN  FIRST  0.5 
SECOND  OF  A  DETECTION  RESPONSE 
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Figure  13  MASKING  VEIL  FOR  ISOLATION  OF 
AN  AREA  DURING  IDENTIFICATION  RESPONSE 
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RESULTS 


The  dependent  measures  available  tor  analysis  include  the  number  of  targets 
correctly  detected,  the  number  of  detected  changes  correctly  identified,  in¬ 
spection  time  required  for  detection,  inspection  time  required  for  identification, 
and  the  number  of  commission  errors  elicited.  These  performance  measures 
are  summarized  for  each  treatment  and  treatment  level  in  Table  VII.  Because 
absolute  ground  truth  (number  of  changes)  was  known,  both  detections  and  identi¬ 
fications  are  expressed  as  percentages  that  are  interpretable  as  indices  of  com¬ 
pleteness. 

To  aid  the  reader  in  integrating  pertormance  levels  across  experimental  con¬ 
ditions,  these  data  are  presented  graphically  in  Figures  14A  through  14G. 

The  first  analysis  performed  was  to  evaluate  the  effects  of  the  various  treat¬ 
ments  on  the  number  of  changes  detected.  This  measure  involves  only  the 
correct  report  of  the  location  of  a  change  and  is  considered  independent  of 
whether  the  subsequent  identification  was  correct.  The  analysis  of  variance  is 
summarized  in  Table  VIII (A)  and  reveals: 

1)  A  significantly  larger  number  (p  less  than  0.01)  of  changes  were  correctly 
detected  with  the  apparent-motion  display  system  (39.0  percent)  than  with 
the  side-by-side  presentation  (17.0  percent). 

2)  Pholoinlerpreters  did  not  detect  a  significantly  greater  or  fewer  number  of 
changes  than  did  the  non-photo  in  lei  prefers. 

3)  The  number  of  changes  correctly  detected  (p  less  than  0.01)  was  signifi¬ 
cantly  intluenced  by  the  contrast  level  of  the  imagery.  Only  13.2  percent 

of  the  changes  were  reported  when  the  film  contrast  was  2u  percent,  whereas 
30.8  percent  were  correctly  detected  with  the  40  percent  contrast  material, 
and  40.0  percent  were  detected  with  the  highest  contrast  (GO  percent) 
imagery.  Duncan’s  New  Multiple  Range  Test  (Edwards,  1900)  was  applied 
and  revealed  significant  performance  differences  between  each  contrast 
level. 

4)  A  significant  interaction  (p  less  than  0.01)  between  display  methods  and 
contrast  level  was  found  to  exist.  By  inspecting  the  curves  presented  in 
Figure  11  (A),  it  can  be  seen  that,  whereas  detection  performance  with  the 
apparent-motion  display  increases  almost  linearly  as  contrast  improves, 
the  performance  levels  with  Hie  s  to-by-side  display  does  not  improve  for 
contrasts  above  40  percent.  Critical  ratios  computed  on  the  performance 
differences  between  successive  contrast  levels  reveal  significant  increases 
(p  less  than  0.01)  with  eacli  improvement  in  contrast  for  the  apparent- 
motion  method  and  a  significant  improvement  only  between  the  20-  and  -1U- 
percenl  contrast  levels  with  the  side-by-side  display. 

Following  each  report  of  a  detected  change,  subjects  were  required  to  identify 
the  object  that  had  been  changed  and  to  specify  the  type  of  change  (i.e.  ,  number, 
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Table  VII 

TABLE  OF  MEANS  FOR  EXPERIMENT  I 
(A) 


PERCENT  OF  TARGETS  CORRECTLY  DETECTED 


20%  CONTRAST 

AM  SxS 

40%  CONTRAST 

AM  SxS 

60%  CONTRAST 
AM  SxS 

Non-P.  1 . 
Group 

17.5  8.3 

42.6  24.1 

63.9  20.4 

P.l. 

Group 

17.5  9.3 

40.7  15.7 

51.8  24.1 

“  MEAN1 

13.15  30.77  40.05 

(B) 

PERCENT  OF  TARGETS  CORRECTLY  IDENTIFIED 

20%  CONTRAST 

AM  SxS 

40%  CONTRAST 

AM  SxS 

60%  CONTRAST 
AM  SxS 

Non-P.  1  * 
Group 

12.0  4.6 

32.4  17.6 

50.0  16.7 

p.i. 

Group 

13,9  7.4 

33.3  14.8 

46.3  20.4 

MEAN1 

9A9 

PERCENT  OF  t 

24.54  33.40 

(0 

)ETECTIONS  CORRECTLY  IDENTIFIED 

20%  CONTRAST 

AM  SxS 

40%  CONTRAST 

AM  SxS 

60%  CONTRAST 
AM  SxS 

Non-P.  1 . 
Group 

68.4  50.0 

76.1  65.4 

78.3  81.8 

P.l. 

Group 

78.9  88.8 

81.8  94.1 

89.3  84.6 

MEAN, 

5  71.52  79.35  83.50 

MEANS 

29.47 

26.52 


MEANS 

22.22 

22.68 


MEANS 

70.0 

86.25 
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Table  VII  TABLE  OF  MEANS  FOR  EXPERIMENT  I  (CONT. ) 

(D) 

MEAN  DETECTION  TIME  PER  PAIR  (SECOND5) 


20%  CONTRAST 

AM  |  SxS 

40%  CONTRAST 

AM  SxS 

60%  CONTRAST 
AM  SxS 

MEANS 

Non-P.l. 

Group 

130.0 

188.2 

161.0 

217.6 

148.2 

200.9 

174.3 

P.l. 

Group 

143.2 

182.5 

130.6 

211.4 

137.3 

186.0 

165.2 

MEANS  161.0  181.1  168.1 


(E) 

MEAN  IDENTIFICATION  TIME  PER  RESPONSE  (SECONDS) 


20%  CONTRAST 

AM  SxS 

_ i 

40%  CONTRAST 

AM  SxS 

60%  CONTRAST 
AM  SxS 

MEANS 

Non-P.l, 

Group 

r 

4.3 

4.6 

9.4 

9.2 

9.6 

7.1 

7.37 

P.l. 

Group 

12.3 

10.6 

! - 

10.1 

6.8 

10.3 

i 

12.3 

10.40 

MEANS  7.95  8.87  9.82 


(F) 

MEAN  INSPECTION  TIME/PAIR  (SECONDS) 


MEANS  169.25  194.60  184.15 
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Table  VII  TABLE  OF  MEANS  FOR  EXPERIMENT  I  (CONT.) 


Table  VIII  SUMMARY  OF  THE  ANALYSES  OF  VARIANCES 

Experiment  I 

(A)  (B) 


DETECTIONS _ IDENTIFICATIONS 


SOURCE  OF  VARIANCE 

df 

ms 

F 

ms 

F 

Between  Groups  (G) 

1 

HHH 

1 ,45 

0.06 

0.32 

Between  Displays  (D) 

1 

84o39** 

91.84 

47.43** 

Between  Contrasts  (C) 

2 

143.00 

82.66** 

56.66 

29.26** 

G  x  D 

1 

0.34 

0.20 

0.17 

0.09 

G  x  C 

2 

0.81 

0.47 

0.27 

0.14 

D  x  C 

2 

20.23 

1 1 .69** 

12.60 

6.51** 

G  x  D  x  C 

2 

O  nn 

z  *  uv 

1.16 

0.75 

0.39 

Error  Variance 

132 

1.73 

— 

1.94 

— 

TOTAL 

143 

(0  (D) 

DETECTION  TIME  IDENT.  TIME 


SOURCE  OF  VARIANCE 

df 

ms 

F 

ms 

1  F  '  1 

Between  Groups  (G) 

1 

6,833 

2.25 

327 

mm 

Between  Displays  (D) 

1 

12,027 

39.59** 

29 

HUH 

Between  Contrasts  (C) 

2 

2,867 

0.94 

42 

0.54 

G  x  D 

1 

1,921 

0.63 

0.34 

0.01 

G  x  C 

2 

3,959 

1.30 

117 

1.50 

D  x  C 

2 

2,324 

0.77 

8 

0.10 

G  x  D  x  C 

2 

102 

0.04 

51 

0.66 

Error  Variance 

132 

3,038 

— 

77 

— 

TOTAL 

143 

(E) 

(F) 

TOTAL  INSP.  TIME 

COMMISSION  ERRORS 

SOURCE  OF  VARIANCE 

df 

ms 

F _ . 

ms 

F 

Between  Groups  (G) 

1 

72,092 

9.51 

1.17 

Between  Displays  (D) 

1  2 

2,581,913 

BS&ililiH 

333.06 

41.12** 

Between  Contrasts  (C) 

2 

280, 566 

1 .20 

50.76 

6.27** 

G  >;  D 

1 

1,308 

0.01 

0.34 

0.04 

G  x  C 

2 

67,954 

0.29 

4.89 

0.60 

D  x  C 

2 

42,622 

0.19 

24.03 

2.97 

G  x  D  x  C 

2 

46,223 

0.20 

0.84 

0.10 

Error  Variance 

132 

228,  731 

— 

8.10 

— 

TOTAL 

143 

*  -  0.01  <  p<  0.05  **  =  p<0.01 
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DETECTED  TARGETS 

CORRECTLY  IDENTIFIED  TARGETS  CORRECTLY  TARGETS  CORRECTLY 

(%)  IDENTIFIED  <%)  DETECTED  (%) 
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MEAN  DETECTION  TIME  PER  MEAN  IDENTIFICATION  TIME  MEAN  INSPECTION  TIME  PER  PAIR 
PAIR  (SECONDS)  PER  RESPONSE  (SECONDS)  (SECONDS) 


MEAN  COMMISSION  ERRORS  PER  PAIR 


CONTRAST  LEVEL  (%) 


Figure  14  PERFORMANCE  AS  A  FUNCTION  OF  CONTRAST 
IN  Ea^RIMENT  I  (CONT) 


size,  or  position).  In  order  to  be  credited  with  a  correct  identification,  a  sub¬ 
ject  must  have  accurately  reported  both  pieces  of  information.  The  analysis  of 
variance  based  on  identification  scores  is  summarized  in  Table  VIII(B).  The 
analysis  shows: 

1)  As  was  true  lor  the  number  of  detections,  the  use  of  the  apparent -mot ion 
display  system  significantly  enhanced  (p  less  than  0,01)  target-change 
identification  performance.  Only  13.7  percent  of  the  target  changes  were 
correctly  identified  when  the  comparative  pairs  were  displayed  side-by-side, 
whereas  31.0  percent  of  the  changes  were  correctly  identified  with  the 
apparent -mot  ion  display. 

2)  No  significant  difference  exists  between  the  two  subject  samples  (non¬ 
photointerpreters  and  photointerprelers)  in  their  ability  to  accurately 
identify  the  type  of  change  and  the  changed  object. 

3)  The  percentage  of  target  changes  correctly  identified  increased  significantly 
(p  less  than  0.01)  with  the  contrast  level  of  the  imagery.  The  mean  per¬ 
centages  of  correctly  identified  target  changes  were  9.  i3,  24.3,  and  33.  4 
for  the  20- ,  40-,  amt  60-percent  contrast  levels,  respectively. 

4)  The  significant  interaction  (p  less  than  0.01)  of  display  method  with  contrast 
level  can  be  seen  in  Figure  14(B)  ami  might  be  attributable  to  the  greater 
sensitivity  of  the  apparent-motion  display  to  changes  in  image  contrast  over 
that  observed  with  the  skle-by-side  display. 

Because  the  identification  of  a  change  cannot  be  made  without  the  change  first 
being  detected,  the  similarity  in  the  detection  anti  identification  performance 
differences  for  the  various  treatments  is  understandable.  A  Pearson  product- 
moment  correlation  computed  between  the  number  of  detections  and  identifications 
for  the  observers  in  each  group  yielded  coefficients  of  0.  87  for  non-photoinlei  - 
prelers  and  0.98  lor  the  oholointerpreters. 

A  second  identification  score,  representing  the  proportion  of  correctly  detected 
changes  that  were  also  correctly  identified,  was  computed  (number  of  correct 
identifications  divided  by  number  of  correct  detections)  for  each  subject.  A 
summary  table  of  the  mean  identification  scores  thus  derived  i,s  presented  in 
Table  VII(C).  These  data  are  illustrated  graphically  in  Figure  14(C). 

Referring  to  Table  VTI(O),  it  can  be  seen  that  photo  interpreters  successfully 
identified  approximately  16  percent  more  detected  changes  than  did  non-pholo- 
interprelers  (86.2  percent  correct  identifications  by  the  former  and  70.0  percent 
identifications  by  the  latter).  Furthermore,  the  proportion  of  delected  changes 
identified  increased  as  the  contrast  of  the  imagery  improved  (71  .5,  79.3,  and 
83.3  percent  with  contrast  material  ol  20,  10,  and  60  percent,  respectively). 

No  indication  appeared  that  identification  performance  was  dependent  on  the 
particular  display  system;  77.4  percent  of  the  detections  were  correctly  identi¬ 
fied  with  the  sidc-by-side  display  and  78.8  percent  were  identified  \v:th  the 
apparent-motion  presentation. 
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The  amount  of  time  required  to  search  for  and  report  the  location  of  each  area 
of  change  is  referred  to  as  detection  time.  The  mean  detection  time  for  the 
various  experimental  conditions  is  found  in  Table  VII(D)  and  is  graphically  pre¬ 
sented  in  Figure  14(D). 

The  analysis  of  variance  for  detection  time  is  summarized  in  Table  VIII(C). 

The  main  effects  include: 

1)  A  significantly  shorter  (p  less  than  0.01)  mean  detection  time  for  material 
viewed  in  apparent  motion  (141.7  seconds  per  pair)  than  that  required  with 
the  side-by-side  display  (197.7  seconds  per  pair).  This  difference  of  almost 
1  minute  (56  seconds)  per  comparison  represents  a  time  saving  of  approxi¬ 
mately  28  percent  by  using  the  apparent-motion  display  system. 

2)  No  significant  differences  in  the  average  detection  time  recorded  for  non- 
photointerpreters  (174.3  seconds  per  comparative  pair)  and  photointerpreters 
(165.2  seconds  per  pair). 

3)  The  average  time  spent  in  detecting  changes  did  not  vary  significantly  as  a 
function  of  the  contrast  level  of  the  imagery.  Mean  detection  times  for  the 
20-,40-,and  60-percent  contrast  levels  were  161.0,  181.1,  and  168.1 
seconds,  respectively. 

The  amount  of  time  required  to  inspect  the  specific  objects  within  an  area  of 
detected  change  and  to  reach  a  decision  about  which  object  has  changed  and  in 
what  way  it  has  changed  (i.e.,  size,  position,  or  number)  is  called  identification 
time.  Although  a  substantial  amount  of  the  identification  task  undoubtedly  was 
completed  simultaneously  with  the  reporting  of  the  detection,  the  relative  dif¬ 
ferences  in  time  scores  achieved  under  the  various  treatments  should  still  reflect 
the  influence  of  the  independent  variables  on  this  aspect  of  the  task. 

Identification-time  data  are  summarized  in  Table  VII(F)  and  presented  graphi¬ 
cally  in  Figure  14(F).  The  results  of  the  analysis  of  variance  are  summarized 
in  Table  VIII(D).  The  major  findings  include: 

1)  Identification  time  was  essentially  the  same  for  both  display  systems  (9.3 
seconds  with  apparent  motion  and  8.4  seconds  for  side-by-side  display). 

2)  Photointerpreters  required  a  significantly  longer  time  (p  less  than  0.05) 
for  each  identification  (10.4  seconds  per  identification)  than  did  the  non- 
photointerpreters  (7.4  seconds  per  identification) .  Although  this  difference 
of  3  seconds  per  identification  is  small  in  absolute  magnitude,  it  does 
represent  a  rale  differential  of  approximately  33  percent.  It  may  be  clue 

to  a  tendency  for  the  experienced  group  to  report  more  detail  than  was 
req u e si ec  1  o  f  t  h  e  m . 

3)  The  differences  in  average  time  required  for  identifications  did  not  reach 
statistical  significance  with  the  three  contrast  levels  investigated.  The 
mean  inspection  limes  were  7.9,  8.9,  and  9.8  seconds  for  the  20-,  10- , 
and  60-percent  contrast  levels,  respectively. 
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Total  inspection  time  is  a  composite  of  detection  lime,  klentilicalion  time,  res¬ 
ponse  time,  interexposure  interval,  and  search  time  following  the  last  reported 
identification  until  signalling  completion  for  each  scene. 

The  average  inspection  time  per  comparative  pair  as  a  function  of  the  display 
methods  and  contrast  levels  for  each  group  is  presented  in  Table  VII(F)  and 
graphically  illustrated  in  Figure  14(F).  The  results  of  the  analysis  of  variance 
are  summarized  in  Table  VIII(E).  The  analysis  shows  that: 

1)  Total  inspection  times  were  significantly  longer  (p  less  than  0.01)  lor 
material  presented  in  side-by-side  format  (205.0  seconds  average  per 
pair)  than  for  material  seen  in  apparent  motion  (160.3  seconds  average 
per  pair). 

2)  No  significant  differences  were  revealed  as  a  function  of  the  experience 
level  of  the  observer.  Photointerpreters  averaged  178.9  seconds  per  pair, 
while  non-photointerpreters  averaged  about  7.5  seconds  longer  per  pair 
with  a  mean  inspection  time  of  186.  4  seconds. 

3)  No  significant  differences  in  inspection  time  were  revealed  as  a  function  of 
the  contrast  level  of  the  photography.  Mean  inspection  times  of  169.2, 
194.6,  and  184.  1  seconds  were  obtained  for  the  20-,  40-,  and  GO-percent 
contrast  levels,  respectively. 

Reports  of  target  changes  where  no  change  was  present  are  commission  errors. 
The  average  number  of  commission  errors  per  comparative  pair  for  each  group 
is  reported  in  Table  VII(G)  (arranged  by  treatments  and  contrast  levels)  and 
presented  graphically  in  Figure  14(G).  The  analysis  of  variance  based  on  the 
number  of  commission  errors  is  summarized  in  Table  VTTT(F).  The  major 
findings  are: 

1)  The  photo  into  rp  ret  or  group  was  not  different  iated  from  the  non-photointer¬ 
preter  group  on  the  basis  of  the  number  of  commission  errors  reported. 

The  latter  group  committed  an  average  of  0.66  errors  per  pair,  whereas 
the  former  group  averaged  0.57  errors  per  pair. 

2)  A  significantly  larger  number  (p  less  than  0.  01)  of  commission  errors  were 
made  when  the  material  was  displayed  in  apparent  motion  (0.87  errors  per 
pair)  as  compared  with  the  number  committed  with  the  side-by-side  presen¬ 
tation  (0.36  per  pair).  In  other  words,  almost  two  and  a  half  times  as  many 
commission  errors  were  made  with  the  apparent-motion  display  as  with  the 
s  id  e-  by  -  s  i  d  e  d  i  sp  1  ay . 

3)  The  rate  at  which  commission  errors  are  made  is  significantly  related 

(p  less  than  0,01)  to  the  contrast  level  of  the  imagery.  The  average  number 
of  commission  errors  per  pair  per  person  was  0.  42,  0.68,  and  0.74  for 
the  20-,  10- ,  and  GO-percenl  contrasts,  respectively. 
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Applying*  Duncan  s  new  multiple  range  test  to  the  performance  differences 
between  contrast  conditions  reveals  that  only  at  the  20-percent  contrast  level 
is  performance  significantly  different  from  that  exhibited  at  either  of  the 
other  levels. 

1)  No  significant  interaction  existed  between  display  method  and  contrast  level. 

An  equal  number  of  target  changes  uf  each  iype  (size,  number,  and  position)  were 
introduced  in  a  controlled,  systematic  fashion.  An  evaluation  of  the  relative 
frequency  with  which  each  typo  of  change  was  detected  is  summarized  in  Table  IX. 


Table  IX  PERCENT  CORRECT  DETECTIONS  FOR  EACH  OF 
THREE  CLASSES  OF  CHANGE 


CHANGE  IN 
NUMBER 

CHANGE  IN 
SIZE 

CHANGE  IN 
POSITION 

28.7 

34.3 

31.5 

m 

14.8 

15.7 

8.3 

PA. 

AM 

27.8 

37.0 

27.8 

Group 

S  S 

17.5 

16.7 

9.3 

MEANS 

22.2 

25.9 

19.2 

Krom  Table  IX,  it  can  be  seen  that,  averaging  across  groups  and  display  methods, 
the  changes  most  difficult  to  detect  were  changes  in  position  (19.2  percent)  and 
changes  in  size'  were  the  most  easily  detected  (25.9  percent)  with  changes  in  num¬ 
ber  being  of  intermediate  difficulty  (22,2  percent). 

Duncan's  now  multiple  range  test  was  applied  to  test  for  performance  differences 
and  failed  to  reveal  a  statistically  significant  difference  between  any  pair  of  mean 
val  ues. 

'file  mean  inspection  times  associated  with  the  comparative  pairs  containing  each 
of  the  three  types  of  change  are  summarized  in  Table  X. 

Table  X  MEAN  INSPECTION  TIME  PER  COMPARATIVE  PAIR  (SECONDS) 
Each  Of  The  Three  Classes  Of  Change 


CHANGE  IN 

CHANGE  IN 

CHANGE  IN 

NUMBER 

SIZE 

POSITION 

Non-P.i. 

AM 

157 

168 

189 

Group 

S  S 

196 

203 

217 

P.I. 

AM 

141 

154 

163 

Group 

S  S 

212 

187 

214 

MEANS 


177 


178 


196 


Although  apparently  longer  average  inspection  times  were  recorded  for  those 
photographs  containing  changes  in  position  (196  seconds)  than  those  obtained 
for  photographs  with  changes  in  size  (178  seconds)  or  number  (177  seconds), 
Duncan's  new  multiple  range  test  did  not  reveal  these  differences  to  be  statisti¬ 
cally  significant. 


CONCLUSIONS 


The  major  puipose  of  this  experiment  has  been  to  determine  the  influence  of 
display  method,  interpreter  experience,  and  image  contrast  on  the  speed, 
accuracy,  and  completeness  of  target -change  detection  and  identification  from 
comparative-cover  aerial  photography. 


Summarizing  briefly  the  main  findings  relative  to  each  of  the  experimental 
variables,  it  has  been  shown  that: 


1)  Significantly  more  changes  were  detected  in  significantly  less  time  in  the 
apparent- mot  ion  display  than  in  the  side-by-side  display.  A  significantly 
larger  percentage  of  the  target  changes  were  correctly  identified  with  the 
apparent-motion  display,  although  this  difference  was  not  evident  for 
identifications  considered  in  proportion  to  detections  of  targets  correctly 
detected,  approximately  an  equal  proportion  was  identified  with  each  display 
system.  Identification  time  did  not  vary  as  a  function  of  the  display  method, 
but  a  significantly  larger  number  of  commission  errors  were  made  with 
the  apparent-motion  format. 


2)  The  two  subject  samples,  dichotomized  on  the  basis  of  their  experience 
with  aerial  photography,  were  differentiated  only  in  the  amount  of  time 
required  lo  make  an  identification  of  the  change:  photointerprelers  required 
significantly  longer  identification  times  than  non-photointerpreters. 

8)  Reducing  the  contrast  of  the  imagery  results  in  a  significant  reduction  in 
the  number  of  changes  detected  and  the  number  of  changes  identified.  The 
apparent-motion  display  system  proved  more  sensitive  to  changes  in  image 
contrast  and  improved  significantly  with  each  increase  in  contrast  level, 
whereas  performance  with  the  side-by-side  display  did  not  improve  for 
contrasts  above  lu  percent.  Neither  detection  nor  identification  time  was 
dependent  on  the  contrast  level  of  the  imagery.  Commission  errors  were 
significantly  higher  with  higher-contrast  imagery. 


Consideration  must  be  given  lo  the.speeific  characteristics  and  limitations  of  the 
stimulus  material,  subject  samples,  measurement  techniques,  and  administration 
procedures  in  extrapolating  the  present  data  or  generalizing  lo  situations  devia¬ 
ting  from  the  experimental  conditions.  Perhaps  the  most  restrictive  element  in 
the  present  study  is  the  nature  ol  the  stimulus  material  employed.  The  two 
photographs  comprising  each  comparative  pair  consisted  of  successive  pictures 
obtained  during  a  single  pass  over  a  target  area.  This  provided  comparative 
imagery  essentially  devoid  of  many  forms  of  ’’visual  noise"  (such  as  would 
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result  from  differences  in  sun  angle,  atmospheric  conditions  or  seasonal  ground 
cover)  typically  present  in  operationally  obtained  comparative  cover  photography. 

Although  all  ground  objects  appeared  to  be  moving  when  viewed  in  the  apparent  - 
motion  display  due  to  the  disparity  in  the  nadir  points  on  each  photograph,  the 
movements  were  essentially  unidirectional  except  lor  those  objects  that  consti¬ 
tuted  target  changes.  Thus,  the  task  confronting  the  observer  in  this  situation 
required  a  discrimination  between  different  types,  amounts,  and  directions  of 
perceived  motion.  Although  not  optimal  for  the  apparent -motion  technique, 
materials  requiring  the  discrimination  of  irregularities  in  a  relatively  uniformly 
changing  field  would  appear  to  be  favorable  to  this  display  method. 

Another  consideration  relating  to  the  test  photography  used  in  this  study  is  that 
the  only  target  changes  represented  in  the  comparative  pairs  were  those  intro¬ 
duced  by  the  artist.  In  this  respect,  all  changes  could  be  classified  as  "artificial. 
The  artificiality  of  the  objects  added,  removed,  or  modified  was  not  discernible 
even  to  critical  experts  specifically  instructed  to  search  for  such  art  work.  This 
degree  of  realism  is  attributable  to  the  skill  and  experience  nf  the  artist  who 
made  the  changes  on  enlarged  prints  and  the  diminution  of  differences  in  texture 
and  definition  with  successive  reproductions  of  the  photographs.  The  procedure 
of  introducing  target  changes  is  highly  desirable  because  it  permits  the  estab¬ 
lishment  of  absolute  ground  truth  for  the  accurate  evaluation  of  the  completeness 
of  information  extraction. 

Caution  should  be  exercised  in  applying  the  present  results  to  predictions  of 
performance  by  currently  practicing  military  photointerpreters.  Photointer- 
prelers  who  participated  in  this  experiment  varied  widely  in  the  amount  and 
type  of  training  and  experience  with  aerial  photography;  also,  none  were  currently 
engaged  in  lull-time  pursuit  of  this  activity.  They  should  be  considered  as 
occupying  some  intermediate  position  along  the  continuum  between  non -photo- 
interpreters  and  currently  active  photointerpreters. 

Although,  in  this  experiment,  non-photointerpreters  did  as  well  as  the  photo- 
interpreters  in  the  detection  of  change  with  either  display  method,  the  photointer¬ 
preters  appeared  to  do  better  in  the  identification  of  the  delected  changes,  par¬ 
ticularly  with  degraded  imagery.  That  the  photointerpreters  required  more 
time  to  make  each  identification  was  apparent  in  the  records  of  the  verbal  identi¬ 
fication  responses  from  each  subject.  They  consistently  responded  with  descrip¬ 
tions  containing  more  information  than  that  required  for  the  simple  identification. 

An  attempt  was  marie  in  this  investigation  to  obtain  discrete  quantitative  measures 
of  the  lime  required  to  make  each  detection  and  the  time  required  to  identify  the 
change.  The  measure  of  detection  time,  from  activation  of  the  shutter  exposing 
the  photographs  to  depressing  the  key  corresponding  to  the  area  of  change,  is 
relatively  straightforward.  The  identification  time,  on  the  other  hand,  should 
not  be  interpreted  as  absolute  because  a  certain  amount  of  the  identification 


process  was  obviously  completed  during  the  detection  portion  of  the  task.  This 
is  particularly  true  in  view  of  the  ’’low  level”  interpretation  (identifying  the  type 
of  change  and  naming  the  object)  required  in  this  study.  Both  measures,  however, 
are  considered  sufficiently  sensitive  to  reflect  significant  effects  of  the  independent 
variables. 


The  results  of  this  experiment  indicate  that  the  apparent -mot ion  display  technique: 
(1)  is  superior  to  the  side-by-side  presentation  for  the  detection  of  changes  (both 
in  completeness  and  time),  (2)  is  comparable  to  the  side-by-side  display  in  iden- 
tilication  of  the  changes  detected  and  identification  time,  and  (3)  is  inferior  to  the 
side-by-side  formal  in  the  number  of  commission  errors  elicited.  It  is  highly 
probable  that  the  advantages  of  the  apparent-motion  display  can  be  further  en¬ 
hanced  and  the  limitations  diminished  by  making  certain  improvements  in  the 
display  apparatus.  Such  changes  might  include  the  modification  of  the  equipment 
to  facilitate  alignment  of  the  two  photographs  and  to  permit  observer  control  over 
the  alternation  rate  of  the  two  images  because  it  appears  that  the  optimal  alter¬ 
nation  rate  for  detection  may  not  be  optimal  for  the  identification  task.  It  is 
recommended  that  continued  research  and  development  of  the  apparent  motion 
display  system  be  conducted  to  further  improve  the  extraction  of  information 
from  aerial  photography. 
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EXPERIMENT  II 


INTRODUCTION 

Experiment  II  was  designed  to  investigate  change-detection  performance  of 
photointerpreters  and  non -photointerpreters,  using  both  the  side-by-side  and 
apparent-motion  displays,  as  a  function  of  various  levels  of  scale,  complexity, 
and  quality  of  the  imagery  being  inspected.  Aerial  photography  representing 
three  scale  ratios  (1:10.000,  1:20,000  and  1:40,000)  and  three  scene  complexities 
(rural,  suburban,  and  industrial-military)  were  reproduced  with  three  imposed 
degradation  levels  and  presented  to  a  group  of  non -photointerpreters  using  two 
display  methods.  The  lowest  and  highest  quality  imagery  at  scales  1:10,000  and 
1:40,000  were  subsequently  presented  to  photo  interpreters  for  inspection  using 
both  display  methods.  Performance  evaluation  included  both  the  relative  accuracy 
of  change  detection  and  the  speed  with  which  the  task  was  accomplished. 


SUBJECTS 


Eighteen  Booing  employees  served  as  the  subject  sample  of  non-photointerpreters. 
Qualifications  for  inclusion  in  this  sample  were:  (1)  "normal"  (corrected  to 
20/20)  vision,  (2)  no  formal  training  or  experience  in  photointerpretation,  and 
(3)  that  they  had  not  participated  in  Experiment  I.  Their  current  job  assignments 
with  The  Boeing  Company  were  predominantly  as  shop  technicians,  laboratory 
technicians,  and  junior  and  senior  research  scientists  in  Lhe  Bioastronautics 
organization. 

The  photointerpreters  for  the  experiment  consists  of  eight  persons  from  within 
The  Boeing  Company  who  had  previously  had  formal  training  or  experience  in 
either  military  or  civilian  photo  interpretation.  A  summary  of  their  backgrounds 
in  photointerpretation  can  be  found  in  Appendix  I.  Five  of  these  subjects  had 
also  served  as  observers  in  Experiment  1. 

APPARATUS 

The  side-by-side  and  apparent-motion  display  system  and  the  push-butt.nn -matrix 
response  system  was  the  same  as  that  described  for  Experiment  I.  All  optical 
components  were  cleaned  and  realigned,  and  the  light  sources  were  replaced 
and  calibrated  before  the  beginning  of  data  collection. 

STIMULUS  MATERIA  I, 

With  the  assistance  of  the  Aeronautical  Charting  and  Information  Center 
(Detachment  I)  in  Washington,  D.C.  .  50(5  9-inch  by  9-inch  aerial  photographs 
were  selected  from  the  Air  Force  records  of  comparative  coverages  of  the 
U.S.  Zone  of  the  Interior  that  were  currently  available  from  the  Air  Force,  the 
Department  of  the  Interior,  or  Die  Department  of  Agriculture.  These  photographs 
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represented  53  separate  missions  flown  over  17  different  target  areas.  A 
descriptive  inventory  of  the  material  received  is  included  in  Appendix  III. 

Upon  receipt  of  the  positive  prints,  temporary  mosaics  were  constructed  for 
the  purpose  of  evaluating  print  quality ,  appropriateness  of  ground  activity,  and 
correspondence  of  flight  tracks  and  image  scale.  Subsequent  to  this  review,  an 
order  was  submitted  to  Aeronautical  Charting  and  Information  Center  for  522 
positive  transparent  prints  that  included  Cl  deletions  from  and  17  additions  to  the 
original  order.  The  center  was  able  to  comply  with  this  request  except  for 
coverage  held  by  the  Department  of  Agriculture.  Since  this  collection  agency 
was  in  the  process  of  relocating  its  photo  library,  it  could  not  make  available 
either  transparent  prints  or  the  negatives  of  the  desired  coverages.  Therefore, 
Boeing  made  negatives  directly  from  the  original  prints.  In  this  way,  it  was 
possible  to  make  the  final  selection  of  the  required  72  comparative  pairs  (eight 
samples  from  each  of  three  backgrounds  —  rural,  suburban,  and  industrial- 
military —  at  the  three  desired  scales  — 1:10,000,  1:20,000,  and  1:40,000)  from 
within  47  separate  mission  coverages. 

After  these  original  9 -inch  by  9-inch  photographs  that  contained  desired  target 
areas  had  been  selected,  negatives  were  made  from  the  transparencies  (except 
in  the  case  of  the  Department  of  Agriculture  coverages  where  negatives  were 
made  from  the  positive  prints) ,  and  enlargements  to  a  20-inch  by  20 -inch  format 
were  prepared.  The  desired  target  areas  within  each  of  the  144  photographs 
(72  comparative  pairs)  were  then  delineated  by  crop  marks  and  a  scale-reference 
line  added.  Target  changes  were  then  introduced  on  one  member  of  each  picture 
pair  (with  a  few  exceptions  where  no  changes  were  introduced)  by  a  skilled 
artist.  These  changes  were  modeled  after  changes  observed  elsewhere  In  the 
photograph  and  were  included  here  in  an  attempt  to  establish  the  validity  of  this 
procedure  for  introducing  targets  on  imagery  where  no  "real’'  targets  exist  or 
where  ground  truth  cannot  be  determined.  Thirty-five  such  targets,  matched  in 
size  and  contrast  with  existing  ’’real"  targets,  were  added  (nine  in  the  1:10,000, 
14  in  the  1:20,000,  and  12  in  the  1:40,000  scale  photographs)  with  the  number 
per  comparative  pair  varying  between  zero  and  three. 

The  final  negatives  of  the  cropped  target  areas  were  then  prepared.  The  scale- 
reference  lines  were  also  included  in  the  periphery  since  these  would  be  required 
in  producing  the  final  4-inch  by  4-inch  transparencies  at  the  desired  scales. 

Of  the  numerous  techniques  used  to  degrade  photographic  quality,  defocus ing 
is  probably  most  often  reported  in  the  literature  since  it  is  perhaps  the  easiest 
and  m ^st  readily  available  means  of  creating  a  blurred  image.  A  number  of 
research  persons,  such  as  Stanley  Ballard,  W.E.K,  Middleton,  and  Glen  A.  Fry 
(Fry,  1957),  have  discussed  the  limitations  of  this  technique.  The  major  limita¬ 
tion  is  that  defocusing  does  not  lend  itself  to  quantitative  specification.  The 
method  used  in  this  experimental  investigation  is  one  recommended  by  Fry  (1957) 
that  lends  itself  to  qualitative  description  and  quantification. 
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A  neutral-density  filter  was  produced  according  to  the  procedure  advanced  by 
Fry  and  as  described  in  detail  in  Appendix  IV.  The  central  density  was  0.48 
with  the  gradient  along  any  diameter  an  approximation  of  the  gaussian  distribu¬ 
tion.  The  deviation  was  in  the  platykurtic  direction,  and  the  4.0  standard 
deviation  point  on  this  curve  was  equated  to  the  entrance  pupil  of  an  Eastman 
EKtanon  (10 -inch  focal  length)  lens.  The  enlarging  iens  and  filter  were  mounted 
in  an  8-inch  by  10-inch  Elwood  enlarger.  The  lens,  filter,  and  optical  path 
were  checked  and  adjusted  to  give  a  homogeneous  blur  in  both  X  and  Y  directions 
through  the  range  of  blur  planned  for  this  study. 

The  4- inch  by  4-inch  negatives  were  used  in  the  Elwood  enlarger  to  make  the 
two  levels  of  blur.  The  procedure  was  to  preset  the  enlarger  at  -2,0  millimeters, 
and  later  -4.0  millimeters,  from  best  focus  and  then  match  the  image  size  to 
that  of  the  original  negative.  The  gaussian  filter  was  placed  over  the  entrance 
pupil,  and  all  prints  were  made  without  changing  the  enlarger.  Calibration  of 
the  condition  was  made  by  printing  a  resolution  chart  on  the  same  transparency 
material  (Type  3  Kodalith)  and  developing  it  in  Dcktol  1:1.  (The  calibration 
target  was  the  USAF  resolution  chart,  developed  under  Contract  Number 
(580-66-SA -10  by  the  Buekbee-Mears  Company,  St.  Paul,  Minnesota.)  This 
print  and  those  for  all  other  image  degradation  conditions  were  examined  with  a 
Model  4  A  ns  co  microdensitometer.  The  resulting  conditions  are  summarized  in 
Table  XI  and  Figure  15.  The  blur  levels  were  selected  to  produce  a  maximum 
practical  range  (without  false  resolution)  on  the  basis  of  appearance. 

Table  X!  PHYSICAL  INFORMATION  RELATIVE  TO  BLUR  CONDITIONS 
Experiment  II  Stimulus  Materials 


MEASURE 

IMAGE  QUALITIES 

HIGHEST 

INTERMEDIATE 

POOR 

QUALITY 

QUALITY 

QUALITY 

Edge  Gradient  (Minimum  Density 
to  Maximum  Density) 

0.31  mm 

0.65  mm 

1.25  mm 

Resolution  (Lines  Per  mm) 

64.0 

1.25 

0.80 

Resolution  (Ground  Resolution 
in  Feet) 

1:10,000  Scale 

0.505 

26.2 

41.0 

1:20,000  Scale 

1.010 

52.4 

82.0 

1:40,000  Scale 

2.020 

104.8 

164.0 

Acutance  GD/(Xg-X^) 

0.0255 

0.0154 

0.0008 

Density  Range 

1.2 

1.2 

1.2 

(ii 


KNIFE 

EDGE 


0.0 


HORIZONTAL  SCALE: 
- 2  INCHES - 

- 1  MM - 


Figure  15  MICRODENSITOMETER  TRACINGS  OF  EDGE  GRADIENTS 
FOR  A  KNIFE  EDGE  AND  THE  THREE  PHOTOGRAPHIC  QUALITIES 
USED  IN  EXPERIMENT  I! 

Ansco  Model  4  Microdensitometer;  2  By  60-Micron  Slit 


These  physical  conditions  describe  the  best  conditions  that  could  prevail  for 
each  blur  level.  Each  photographic  negative  had  its  own  physical  characteristics 
and,  when  printed  through  the  Elwood  enlarger  with  the  selected  degree  of 
defocus  and  the  gaussian  filter,  was  made  poorer  by  the  imposed  blur.  The 
resulting  image  quality  could  be  no  better  than  the  values  given  in  Table  XI,  and 
to  the  extent  that  the  original  is  less  than  perfect,  this  quality  will  be  less  than 
the  tabulated  values.  The  highest  and  lowest  image  qualities  for  each  of  the 
three  scales  and  three  complexities  are  illustrated  in  Figures  l(i,  17,  and  18. 

TRAINING 

All  18  subjects  constituting  the  non-photointerpreter  population  sample  were 
given  the  training  program  described  for  Experiment  I.  Photointerpreters  who 
had  not  participated  in  Experiment  I  were  likewise  given  the  complete  training 
program  described  for  that  experiment.  For  the  five  photointerpreters  who  had 
served  as  observers  in  Experiment  I,  a  brief  review  of  the  basic  operating  pro¬ 
cedures  of  the  display  system  and  the  response  panel  was  provided.  They  were 
then  presented  three  of  the  '’high  visual  noise"  training  slides  under  each  of  the 
presentation  methods  to  re  acquaint  their,  with  how  aerial  photography  appears 
when  so  displayed  and  to  allow  them  to  remaster  the  use  of  the  response  panel 
for  indicating  target  locations.  Immediate  feedback  on  their  performance  was 
provided  after  each  response  and  again  at  the  completion  of  each  of  the  three 
comparative  pairs. 


INSTRUCTIONS 

In  addition  to  the  general  instructions  describing  the  purpose  of  the  research 
program,  the  importance  of  the  contributions  being  made  by  the  observer,  and 
the  performance  criteria  being  used  in  the  evaluation,  the  following  task-specific 
instructions  were  given. 

"The  task  which  you  are  being  asked  to  perform  next  differs  significantly 
from  that  which  you  were  asked  to  do  in  the  training  session  (and  in  the 
preceding  Experiment)  in  one  respect.  With  the  preceding  material  you 
were  required  to  report  the  location  of  each  and  every  change  which 
occurred  between  the  two  photographs  and  also  to  identify  the  object  which 
had  changed  and  the  type  of  change  which  had  occurred  (whether  it  was  a 
change  in  size,  number,  or  position).  Since  among  the  photographs  you  will 
be  viewing  next  are  some  representing  what  we  call  'high  activity  areas’ 
such  as  New  York  City  and  since  the  elapsed  time  between  the  first  and  the 
second  photograph  may  be  as  much  as  several  years,  the  job  of  locating 
and  identifying  all  of  the  changes  in  such  an  area  becomes  prohibitive  within 
any  reasonable  demands  on  your  valuable  time.  To  make  the  task  a  little 
more  reasonable  we  will  designate  a  specific  target  class  before  presenting 
each  set  of  pictures  and  you  will  be  required  to  locate  and  report  only  those 
changes  which  have  occurred  to  objects  falling  within  the  specified  target 
class.  For  instance,  1  may  tell  you  to  look  only  for  changes  in  large, 


Figure  16  ILLUSTRATIONS  OF  HIGH  AND  LOW  IMAGE  QUALITIES 
WITH  RURAL  TERRAIN  1  :  10,000  Scale 


Figure  17  ILLUSTRATIONS  OF  HIGH  AND  LOW  IMAGE  QUALITIES 
WITH  SUBURBAN  TERRAIN  1  :  20,000  Scale 


Figure  18 


.LUSTRATION  OF  HIGH  AND  LOW  IMAGE  QUALITY 
H  INDUSTRIAL  TERRAIN  1  :  40,  000  Scale 
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NON-RESIDENTIAL  BUILDINGS,  in  which  case  you  would  not  report  the 
addition  of  new  homes  in  the  area  but  you  would  report  the  addition  of  a  new 
warehouse  or  factory.  Other  examples  of  the  types  of  target  classes  you 
may  be  asked  to  look  for  would  be  ROADS  AND  HIGHWAYS,  DOCK 
ACTIVITY  in  a  harbor  scene,  or  perhaps  ANY  MAN-MADE  OBJECT  in  the 
case  of  pictures  of  desolate  rural  terrain. 

’’You  will  still  be  required  to  tell  us  in  what  way  the  object  you  ai'e  reporting 
has  changed  (that  is,  whether  it  is  a  change  in  size,  position,  or  number). 
Your  verbal  description  of  the  specific  object  within  the  designated  target 
class  that  you  have  detected  as  changed  will  aid  us  in  obtaining  an  accurate 
measure  of  your  performance. 

"The  criteria  to  be  applied  in  evaluating  your  performance  will  be 
ACCURACY,  COMPLETENESS,  AND  SPEED.  Therefore,  you  should  be 
certain  that  you  have  detected  all  of  the  changes  falling  in  the  target  class 
specified  (COMPLETENESS),  that  you  have  accurately  identified  the  object 
changed  and  how  it  has  changed  (ACCURACY),  and  that  you  are  working  as 
rapidly  as  possible  without  sacrificing  either  accuracy  or  completeness. 

"Arc  there  any  questions  before  we  begin ?’’ 

DESIGN 

The  basic  experimental  design  is  represented  schematically  in  Figure  19.  Each 
of  the  18  non -photo interpreters  was  randomly  assigned  to  one  of  three  groups  of 
six  subjects  each.  All  members  of  a  given  group  viewed  only  the  material  repre¬ 
senting  one  of  the  three  scales —  1:10,000,  1:20,000,  or  1:40,000.  Within 
groups,  each  subject  inspected  18  different  comparative  pairs;  two  comparative 
pairs  represented  each  of  the  three  background  complexities  reproduced  at  each 
of  the  three  levels  of  image  quality.  Half  wci’c  viewed  side  by  side  and  the  other 
half  were  viewed  in  apparent  motion. 

Counterbalancing  for  presentation  order  effects  was  achieved  by  arbitrarily 
designating  half  of  the  subjects  within  each  group  to  be  presented  half  of  the 
imagery  in  side-by-side  format  first  and  then  the  remaining  half  of  the  material 
in  apparent-motion  format;  the  other  half  of  the  subjects  viewed  the  material  in 
the  reverse  display  order. 

To  avoid  any  systematic  biasing  of  the  results  due  to  the  order  in  which  the 
three  background  complexities  were  presented,  each  of  six  possible  permuta¬ 
tions  of  the  three  background  orders  were  experienced  by  each  observer — -one 
order  for  each  of  the  blur/display-method  (three  by  two)  conditions. 

Each  of  the  six  subjects  within  a  group  was  assigned  to  one  of  the  six  possible 
presentation  orders  for  the  three  image  qualities  so  that,  within  each  group,  a 
complete  Latin  square  was  represented. 
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DISPLAY  METHOD  DISPLAY  METHOD 


Nonphotointerpreter  Group  (N  =  18) 


FOR  EXPERIMENT  II 


The  eight  photointerpreters  each  inspected  24  separate  comparative  pairs. 
Half  were  viewed  side  by  side  and  half  with  apparent  motion.  Of  the  12  areas 
seen  under  either  display  system,  one  represented  each  of  the  background/ 
scale/image-qualit}'  (three  by  two  by  two)  conditions. 

Complete  counterbalancing  for  effects  attributable  to  the  presentation  order  of 
the  two  image  qualities  and  the  two  scales  was  achieved  within  subjects.  Half 
of  the  subjects  used  the  side-by-side  display  first;  the  other  half  used  the 
apparent-motion  display  first. 


RESULTS 


Because  absolute  ground  truth  for  the  stimulus  material  used  in  this  investiga¬ 
tion  was  not  available,  the  following  procedure  was  applied  to  establish  a  per¬ 
formance  criterion  for  evaluating  the  treatment  effects: 

All  of  the  responses  (indicating  the  detection  of  change)  made  by  either 
group  of  subjects  were  itemized  and  pooled.  Added  to  this  list  of  respon¬ 
ses  were  responses  contributed  by  members  of  the  research  team  as  they 
prepared  and  reviewed  the  comparative  pairs.  When  all  subjects  had  in¬ 
spected  a  specific  picture  pair,  each  of  the  responses  elicited  was  evalua¬ 
ted  for  correctness  by  reference  to  the  enlarged  photographs  (20  inches  by 
20  inches)  used  in  preparing  the  final  transparencies  for  the  final  evalua¬ 
tion.  Thus,  each  individual's  performance  became  relative  to  the  total 
number  of  changes  detected  by  the  composite  group,  and  only  those  changes 
detected  by  at  least  one  of  the  observers  and  verified  on  the  enlargements 
were  included  in  the  analysis  of  completeness.  Changes  reported  but 
not  verified  on  the  enlargements  were  tabulated  as  errors  of  commission. 

A  commission  error  was  also  recorded  when  a  subject  reported  a  change 
that  involved  an  object  belonging  to  some  target  class  other  than  that  which 
he  had  been  instructed  to  search  for  and  report. 

Significant  differences  in  accuracy  as  a  function  of  scale  were  not  anticipated 
because  the  scoring  criterion  was  derived  from  the  composite  group  performance 
at  each  scale. 

Tables  XII  (A)  and  (B)  summarize  the  percentage  of  correct  detections,  mean 
inspection  time  per  comparative  pair,  and  the  mean  number  of  commission 
errors  per  pair  for  each  of  the  treatments  and  treatment  levels.  These  data 
are  presented  graphically  in  Figures  20  through  26. 

The  nature  of  the  stimulus  material  available  for  this  experiment  required  that 
the  search  be  limited  to  certain  specifically  designated  target  classes.  Because, 
in  some  instances,  only  a  single  type  of  target  was  assigned,  the  identification 
report  could  not  be  distinguished  from  detection.  Furthermore,  the  type  of 
change  occurring  between  the  two  photographs  of  any  pair  predominantly  con¬ 
cerned  changes  in  number  (additions  or  deletions),  and  only  rarely  were  changes 
in  size  or  position  included.  These  factors,  coupled  with  the  high  correlation 
found  in  Experiment  I  between  detections  and  identifications  (r  =  0.94  for  com¬ 
bined  groups),  seemed  to  justify  the  exclusion  of  identification  data  from  further 
analysis. 

Looking  first  at  the  data  obtained  from  non -photointerpreters,  the  following 
results  were  obtained: 

1)  The  percentage  of  targets  correctly  detected  was  essentially  the  same 
(53, 1  percent  as  compared  with  47.  7  percent)  for  both  display  methods. 
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Table  XII  PERFORMANCE  MEANS  FOR  ALL  CONDITIONS 
OF  EXPERIMENT  II 


(A)  Non-P.l. 


DISPLAY 

METHOD 

IMAGE 

QUALITY 

PHOTO 

SCALE 

SCENE 

BACKGROUND 

%  Correct 
Detections 

AM  =  53.1 

SS  =  47.7 

Hi  -  59.6 

Md  =  56.4 

Lo  =  41.3 

10K  =  57.7 

20K  =  64.8 

40K  -  28.8 

Rur .  -■  53.1 

Sub.  =  46.9 

Ind .  =  51.3 

Mean  Time 
Per  Pair 
(Seconds) 

AM  =  142.5 

SS  =  167.1 

Hi  =  159.7 

Md  =  154.4 

Lo  =  150.4 

10K  =  117.9 

20K  -  163.0 

40K  =  183.6 

Rur.  =  138.1 
Sub.  =  151.4 
Ind.  a  175.0 

Commission 
Errors  Per 
Pair 

AM  =  1 .68 

SS  =  1.51 

Hi  =  1.66 

Md  =  1.44 

Lo  =  1,68 

1  OK  =  1.06 

20K  =  2.07 

40K  =  1.64 

Rur.  =  1.56 

Sub.  =  1.52 

Ind.  =  1.70 

10K  =  1:10,000 


(B)  P.l. 


DISPLAY 

METHOD 

IMAGE 

QUALITY 

PHOTO 

SCALE 

SCENE 

BACKGROUND 

%  Correct 
Detections 

AM  =  50.3 

SS  =  42.1 

Hi  =  58.5 

Lo  =  33.9 

10K  =  50.0 

40K  =  42.5 

Rur.  =  46.5 

Sub.  =  43.6 

Ind.  =  48.6 

Mean  Time 
Per  Pair 
(Seconds) 

AM  =  128.1 

SS  =  159.6 

Hi  =  145.0 

Lo  =  142.7 

10K  -  120.9 

40K  =  166.7 

Rur.  =  124.5 
Sub.  =  158.9 
Ind.  -  148.1 

Commission 
Errors  Per 
Pair 

AM  =  0.93 

SS  =  1. 18 

Hi  =  1.01 

Lo  =  1.09 

10K  =  0.81 

40K  =  1.29 

Rur.  =  0.91 

Sub.  =  1.19 

Ind.  =  1.06 

T2 
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0.01 


0.05 


1:10,000  1:20,000  1:40,000 


Figure  20  PERCENT  CORRECT  DETECTION 
AS  A  FUNCTION  OF  SCALE  IN  EXPERIMENT  II 


70 


Non-P. 


UJ 

Od 

2 .60  r 

Z) 

1 — 

2.40  - 

U 

Cl. 

2.20  - 

OC 

UJ 

o_ 

2.00  - 

CO 

OC 

1  .80  - 

o 

O' 

1.60  - 

QC 

UJ 

1.40  - 

2 

1,20  - 

o 

CO 

CO 

1 .00  - 

:> 

:> 

0.80  - 

0 

0.60  - 

z 

< 

UJ 


2 


L_ 


UJ 


oc 

ID 

1.60 

h— 

u 

1.50 

a. 

1 .40 

OJ 

a. 

1.30 

co 

o 

CN 

CtL 

o 

OC 

OC 

1.10 

UJ 

z 

o 

o 

g 

0.90 

CO 

CO 

0.80 

£ 

£ 

0.70 

o  1 

Z  L_ 

< 

UJ 

2 


Figure  22  MEAN 


1:10,  000 


1:20,  000 

SCALE 


1 :40,  000 


—  APPARENT  MOTION  X  -  LOW  BLUR 

. SIDE  BY  SIDE  O  -  MED  BLUR 

□  -  HIGH  BLUR 


DUMBER  OF  COMMISSION  ERRORS  PER 
RATIVE  PAIR  FROM  NON-P.  I.AND  P.l. 
/ERS  AS  A  FUNCTION  OF  SCALE,  IMAGE 
Y,  AND  DISPLAY  METHOD 


1:10,  000 


1:20,000 

SCALE 


1 :40,  000 


X  -  INDUSTRIAL 
O  -  SUBURBAN 
□  -  RURAL 


Figure  23  MEAN  TIME  PER  COMPARATIVE  PAIR  AS  A  FUNCTION 
OF  SCALE  FOR  THREE  TERRAIN  COMPLEXITIES,  NON-P.I. 
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Figure  24  BAR  GRAPH  OF  MEAN 
PERCENT  DETECTIONS  AS  A  FUNCTION  OF  DISPLAY  METHOD, 
SCALE  AND  IMAGE  QUALITY  Non-P.l.  And  P.l. 
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Figure  25  BAR  GRAPH  OF 
DETECTION  TIME  AS  A  FUNCTION  OF  DISPLAY 
METHOD,  SCALE,  AND  IMAGE  QUALITY  Non-P.l.  And  P.l. 
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MEAN  COMMISSION  ERRORS  PER  PICTURE 


The  difference  of  5.4  percent  favoring  the  apparent- motion  display  was 
not  significant  (see  Table  XIII), 

2)  The  percentage  of  targets  correctly  detected  was  significantly  related 
(p  less  than  0.  01)  to  the  quality  of  the  imagery  being  viewed  (see  Table 
XIII).  Application  of  DuncanTs  new  multiple  range  test  (Edwards,  1950) 
shows  that  the  performance  levels  achieved  at  each  level  of  image  quality 
(59.6,  50.4,  and  41.3  percent  for  the  high,  medium,  and  low  qualities, 
respectively)  is  significantly  different  (p  less  than  0.  01)  from  that  at  each 
of  the  other  levels. 

3)  The  percentage  of  targets  correctly  detected  did  not  vary  systematically 
with  the  scale  of  the  imagery  (57.  7  percent  at  1:10,  000,  64.8  percent  at 
1:20,  000,  and  28.  8  percent  at  1:40,  000).  The  difference  between  the  per¬ 
formance  level  exhibited  with  the  1:40,  000  imagery  and  that  attained  with 
the  1:10,  000  and  1:20,  000  material  was  considerable  (28.9  and  36.  0  per¬ 
cent,  respectively),  and  was  significant  at  the  0.01  level  of  confidence 
(see  Table  XIII). 

4)  The  percentage  of  targets  correctly  detected  was  not  related  to  the  com¬ 
plexity  of  the  background  contained  in  the  photographs.  For  rural  material, 
53.1  percentof  the  targets  were  detected;  for  suburban  material,  46.9  per¬ 
cent;  and  for  the  high-complexity  industrial  background,  51.2  percent. 

5)  The  non -photointerpreters  required  significantly  less  time  to  inspect  each 
pair  of  photographs  when  presented  in  apparent  motion  than  they  did  when 
the  photos  were  displayed  side  by  side  (p  less  than  0.  01,  see  Table  XIII). 

An  average  of  167,1  seconds  was  required  to  search  each  pair  in  the  side- 
by-side  format;  an  average  of  only  142.5  seconds  was  required  for  the  task 
with  the  apparent-motion  display  — a  saving  of  about  25  seconds  per  com¬ 
parison,  or  about  15  percent. 

6)  The  amount  of  time  required  to  search  each  comparative  pair  did  not  change 
significantly  as  the  quality  of  the  imagery  was  varied  (see  Table  XIII). 

Mean  inspection  times  of  159,7,  154.4,  and  150.4  seconds  were  recorded 
for  the  high-,  medium-,  and  low-quality  imagery,  respectively.  If  the 
small  but  systematic  differences  are  indicative  of  a  trend,  it  would  suggest 
that  non-photointerpreters  spend  more  time  on  the  task  as  image  quality 
improves. 

7)  Although  there  is  a  strong  indication  that  mean  inspection  time  increases  as 
the  scale  of  the  photography  becomes  smaller  (117,9  seconds  with  1:10,  000- 
scale  photographs,  163.0  seconds  with  1:20,  000-scale  photographs,  and 
183.6  seconds  with  1:40,  000-scale  photographs),  the  variance  within  and 
between  groups  was  sufficient  to  make  it  impossible  Lo  reject  the  null 
hypothesis  with  acceptable  confidence  (see  Table  XIII). 

8)  The  analysis  reveals  a  significant  time  difference  (p  less  than  0.01)  asso¬ 
ciated  with  the  background  complexity  (rural,  suburban,  or  industrial) 
contained  in  the  imagery  (see  Table  XIII).  Application  of  Duncan's  new 
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multiple  range  test  to  the  differences  between  backgrounds  reveals  that  the 
mean  time  required  to  inspect  rural  and  suburban  photographs  (138  and  151 
seconds,  respectively)  did  not  differ  significantly  from  each  other.  The 
average  of  175  seconds  required  to  search  the  industrial  scenes  was  signi¬ 
ficantly  longer  (p  less  than  0.  01)  than  that  used  on  either  the  rural  or  the 
suburban  material. 

9)  A  significant  background /scale  interaction  (p  less  than  0.  05)  was  revealed 
(see  Table  XIV).  Duncan’s  new  multiple  range  test  was  applied  to  the  mean 
time  scores  for  each  of  the  background/scale  conditions  to  determine  the 
significant  groupings.  Figure  23 contains  the  time  scores  plotted  for  each 
background  at  each  scale.  The  three  circles  areas  represent  the  condi¬ 
tions  yielding  similar  performance  levels.  This  illustration  shows  that 
background  is  a  primary  determiner  of  performance  only  at  a  scale  of 
1:20,  000. 

10)  No  significant  differences  exist  between  the  mean  number  of  commission 
errors  as  a  function  of  the  display  method,  quality  of  the  imagery,  scale 
of  the  photography,  or  type  of  background  contained  in  the  scenes. 

Performance  data  of  the  photointerpreters  show: 

1)  The  percentage  of  targets  correctly  detected  was  essentially  Lhe  same  with 
both  display  methods  (50.  3  percent  with  apparent  motion  and  42. 1  percent 
with  side-by-side  display). 

2)  A  significantly  larger  proportion  (p  less  than  0.01)  of  the  targets  was 
correctly  detected  with  the  high-quality  imagery  (58.5  percent)  than  with 
the  degraded  imagery  (33.9  percent).  The  magnitude  of  this  difference 
(about  25  percent)  is  comparable  to  the  difference  of  18  percent  found  be¬ 
tween  these  two  conditions  for  the  non -photointerpreters. 

3)  The  percentage  of  targets  correctly  detected  with  photographs  representing 
a  scale  of  1:10,  000  was  50.  0,  and  with  photographs  representing  a  scale  of 
1:40,  000,  42.5  percent  were  reported.  The  7.5  percent  of  additional  tar¬ 
gets  detected  with  the  larger -scale  photography  was  not  statistically 
significant  (see  Table  XTV). 

4)  The  complexity  of  the  background  did  not  significantly  affect  detection  per¬ 
formance.  Percentages  of  detections  were  40.5,  43.0,  and  48.0  for  the 
rural,  suburban,  and  industrial  backgrounds,  respectively. 

5)  Mean  inspection  time  for  each  compai'ative  pair  was  significantly  longer 
(p  less  than  0.  05)  with  the  side-by-side  display  than  with  the  apparent- 
motion  display.  The  saving  of  about  31  seconds  per  inspection  (159.6 
seconds  with  side  by  side  as  opposed  to  only  128.  0  seconds  with  apparent 
motion)  was  nearly  the  same  in  magnitude  as  the  25-second  saving  found 
with  the  non-photointerpreters. 

6)  Mean  inspection  time  did  not  vary  with  the  quality  of  the  imagery:  145.  0 
seconds  were  required  with  maximally  degraded  imagery,  and  142.7  seconds 
with  the  highest-quality  material. 
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7)  The  scale  of  imagery  was  a  significant  determinant  (p  less  than  0.  01)  of 
the  time  spent  inspecting  each  pair  of  photographs  (see  Table  XIV).  Sub¬ 
jects  averaged  about  0,  75  minute  longer  inspecting  the  smaller-scale 
photography  (166.7  seconds  average)  than  when  viewing  the  larger-scale 
photography  (120.9  seconds  average). 

8)  Mean  inspection  time  per  comparative  pair  was  not  significantly  related 

to  the  background  complexity  (rural,  124.5  seconds;  suburban,  158.9  seconds; 
and  industrial,  148.  0  seconds). 

9)  The  mean  number  of  commission  errors  per  comparative  pair  was  not 
significantly  affected  by  the  display  method  (0.93  for  apparent  motion  and 
1.18  for  side  by  side),  by  the  quality  of  the  imagery  (1.  01  for  high-quality 
matex*ial  and  1.  09  for  low-quality  photographs),  or  by  the  background  com¬ 
plexity  (rural,  0.91;  suburban,  1.19;  industrial,  1.06), 

10)  A  significantly  greater  number  (p  less  than  0.  05)  of  commission  errors  were 
made  with  the  1:40,  000-scale  photographs  as  compared  with  the  commission 
response  recorded  with  the  1:10,  000-scale  materials.  Approximately  one 
additional  commission  error  was  made  for  every  two  comparative  pairs 
with  the  smaller-scale  photography  (0.81  commission  errors  per  pair  at 
1:10,000  as  compared  to  an  average  1.29  commission  errors  with  1:40,  000- 
scale  imagery). 

A  direct  comparison  between  the  performance  of  photo  interpreters  and  non- 
photointerpreters  was  not  possible  because  the  treatments  and  treatment  levels 
experienced  by  the  subjects  of  each  group  were  not  the  same.  Specifically,  the 
major  differences  between  the  presentations  to  the  two  groups  are  that,  although 
the  stimulus  parameters  used  were  the  same  for  both  groups,  the  photointerpreters 
received  only  two  of  the  three  treatment  levels  of  image  quality  and  scale.  Further¬ 
more,  they  each  participated  under  all  treatments  and  treatment  levels,  whereas 
each  subject  in  the  non-photointerpreter  group  experienced  all  treatment  levels 
except  scale,  for  which  they  observed  only  one  of  the  three  treatment  levels. 

A  limited  comparison  can  be  made,  however,  for  treatment  levels  common  to 
both  groups.  Bartlettrs  test  for  the  homogeneity  of  variance  between  groups 
was  applied  to  each  of  the  dependent  measures.  Group  variances  did  not  differ 
significantly  for  cither  detections  (/3  =  2.99)  or  time  {0 ~  3.31).  Then,  test 
ratios  were  computed  to  test  for  significance  between  means  of  the  two  groups. 

None  of  the  test  ratios  revealed  significant  differences.  This  is  summarized 
in  Table  XV.  Apparent-motion  and  side-by-side  methods  were  pooled  for  this 
analysis. 


Table  XV  SIGNIFICANCE  OF  THE  DIFFERENCES 
BETWEEN  EXPERIENCE  LEVEL  MEANS 


%  CORRECT  DETECTIONS 

NON-P.l. 

d 

t 

o  o 

o  o 
o  o 
o  o 

M  v 

57.7  39.3 

28.8  30.1 

1 

8.1 

13.6 

1.31 

0.91 

H 

MEAN  TIME  PER  PAIR  (SECONDS) 

SCALE 

NON-P.l. 

P.l. 

d 

t 

P* 

‘-n  — 
o  o 

o  o 
o  o 
o  o 

M  a 

117.9  51.9 

183.6  79.6 

M  o 

120.9  52.8 

166.7  77.0 

2.1 

17.2 

0.63 

1.41 

NS 

NS 

*  For  df  =  156:  p  0.05  =  1.97 

p  0.01  =  2.60 


Bartlett's  test  for  homogeneity  of  variance  revealed  that  the  non-photointerpreter 
population  was  significantly  less  variant  (o*  =  0,  8G)  than  the  experienced  group 
( <7  ^  =  2,17)  in  the  number  of  commission  errors  made  (  f3  =  40.  08,  p  less  than 
0.01).  Thus,  a  nonparametric  median  test  was  applied  to  test  the  differences 
between  the  group  medians.  The  resultant  chi-squares  of  0,  028  for  the  1:10,  000- 
scale  material  and  0.73  for  the  1:40,  000-scale  material  were  not  significant. 


CONCLUSIONS 


Experiment  II  was  designed  to  test  the  relative  effectiveness  of  the  two  display 
methods —  apparent  motion  and  side  by  side  —  for  target-change  detection  by 
photointerpreters  and  non-photointerpreters.  The  latter  group  veiwed  the 
comparative-cover  photography  at  three  scales  (1:40,000,  1:20,000,  and 
1:10,  000)  and  three  levels  of  image  quality  (blur).  The  photointerpreters  saw 
only  the  highest  and  lowest  levels  of  scale  and  blur.  The  photography  for  both 
groups  contained  an  equal  number  of  samples  of  three  types  of  background: 
industrial,  surburban,  and  rural.  The  performance  measures  obtained  were 
correct  detections,  false  detections  (errors  of  commission),  and  time  required 
for  the  inspection  of  each  pair. 

The  apparent-motion  method  was  significantly  faster  only  with  respect  to  inspec¬ 
tion  time  (see  Tables  XIII  and  XIV).  This  was  true  for  both  groups.  The  apparent- 
motion  method  yielded  more  correct  detections  from  both  groups,  but  the  differ¬ 
ence  was  not  statistically  significant. 

Blur  was  a  significant  variable  in  the  percentage  of  correct  detections:  fewer 
targets  were  detected  with  each  level  of  degradation.  Although  the  differences 
in  inspection  time  as  a  function  of  blur  were  not  statistically  significant,  the 
values  obtained  are  ordered  in  correspondence  to  the  image  quality.  This  may 
mean  either  that  there  are  fewer  visible  changes  with  increased  blur  (thus 
requiring  less  inspection  time)  or  that  the  observers  are  less  inclined  to  search 
the  pictures  when  image  quality  is  poor.  It  might  be  a  combination  of  both  things. 
However,  it  is  possible  that  performance  would  be  different  in  an  operational 
situation  that  demanded  an  exhaustive  search  with  low-quality  photography. 

Photographic  scale  presents  some  unique  difficulties  for  the  systematic  investi¬ 
gation  of  its  effects  on  target-change  detection.  The  scale  of  a  photograph  can¬ 
not  be  changed  without  changing,  at  the  same  time,  the  value  of  some  other 
physical  parameter.  If  a  photograph  of  1:20,  000  scale  is  enlarged  to  1:10,  000 
scale,  either  the  format  size  must  be  increased,  or  the  ground  coverage  must 
be  reduced.  The  same  format  size  was  maintained  in  Experiment  II  because: 

(1)  constant  format  size  was  judged  to  be  more  typical  of  the  operational  situ¬ 
ation,  and  (2)  it  appeared  desirable  to  keep  the  field  of  visual  search  constant. 

The  effect  of  this  restriction  is  to  allow  ground  coverage  to  vary  with  scale 
and,  along  with  it,  the  number  ot  changes  occurring  in  the  photograph.  The 
scale  in  which  the  terrain  is  represented  is  thus  not  l he  sole  determiner  oi  the 
numerical  values  of  the  response  measures. 

For  both  photointerpreters  and  non-photointerpreters,  inspection  time  was  in¬ 
versely  related  to  scale;  variance  due  to  this  factor  reached  significance  at  the 
99-percent  level  of  confidence  for  the  former  group.  (Strictly  speaking,  there 
is  one  chance  in  100  that  the  observed  difference  is  a  normal  deviation  in  a  single 
stimulus  population.  Differences  in  "scale"  refer  to  whatever  changes  occurred 
as  attempts  were  made  to  manipulate  this  dimension  of  the  photography.) 

87 


Preceding  Page  Blank 


The  difference  in  errors  of  commission  was  statistically  significant  for  scale 
in  the  photointerpreter  group,  and  the  difference  was  in  the  expected  direction. 
This  group  also  evidenced  an  expected  drop  in  the  percentage  of  correct  detec¬ 
tions  with  the  smaller  scale,  although  this  difference  was  not  significant  at  the 
0.  05  level. 

With  the  non-photointerpreter  group,  the  percentage  of  correct  detections  and 
frequency  of  false  detections,  although  not  significant,  corresponded  to  predic¬ 
tions  for  the  highest  and  lowest  scales,  but  not  the  intermediate  scale.  There 
is  no  way  to  determine  whether  this  discrepancy  between  predictions  and  observa¬ 
tions  would  have  occurred  with  the  photo  interpreters  because  they  were  not  pre¬ 
sented  the  1 :20,  000-scale  photography. 

The  difficulties  associated  with  varying  the  scale  of  the  photography  are  reflected 
in  the  number  of  target  changes  occurring  with  the  different  scales  used  in  this 
experiment.  These  changes  were  39,  58,  and  82  for  the  1:10,000,  1:20,000, 
and  1:40,  000  scales,  respectively,  in  the  pairs  presented  to  the  non-photointer¬ 
preters.  For  photointerpreters,  there  were  60  changes  in  the  1:10,  000-scale 
pictures  and  102  in  the  1:40,  000-seaie  material.  Because  of  these  differences 
inextricably  tied  in  with  scale,  the  interpretation  of  the  data  is  Less  straight¬ 
forward  than  it  might  otherwise  have  been.  For  example,  Table  XII  shows  that 
the  non-photointerpreters  took  a  little  less  than  two  thirds  as  long  to  inspect  the 
1:10,  000-scale  photography  as  they  did  for  that  seen  at  1:40,  000  scale.  How¬ 
ever,  in  spite  of  Lhe  longer  time  with  the  smaller  scale,  these  observers  are 
spending  less  time  with  this  scale  if  we  consider  inspection  time  in  terms  of 
the  number  of  target  changes  represented  in  each  scale.  In  these  terms,  they 
are  spending  only  three  fourths  as  much  time  with  the  1:40,  000  material  as  with 
the  1:10,  000  material;  the  situation  considered  in  this  way  is  the  reverse  of  what 
it  appears  to  be  in  Table  XII. 

The  percentage  of  correct  detections  for  these  two  scales  in  the  same  table  shoWvS 
that  close  to  twice  the  percentage  was  obtained  for  the  1:10,000  scale  as  for  the 
1 :40,000  scale.  The  actual  number  of  changes  detected  is  120  for  the  former 
and  136  for  the  latter,  a  larger  number  for  the  smaller  scale  but  a  smaller 
percentage,  by  half.  Further  consideration  of  detections  in  terms  of  the  number 
reported  per  unit  time  reveals  that  performance  with  the  smaller  scale  (1:40,  000) 
is  close  to  70  percent  as  good  as  that  with  the  larger  scale  (1:10,  000)  instead  of 
only  50  percent  (see  Figure  27). 

An  additional  problem  arises  with  the  generation  of  photographic  stimuli  of  dif¬ 
ferent  scales  that  must  be  done  in  the  laboratory  rather  than  originally  photo¬ 
graphed  at  the  desired  scale.  The  problem  is  that  if  the  scale  must  be  increased 
Jrom  lhal  of  the  original,  a  loss  occurs  in  resolution,  and  this  loss  might  in¬ 
crease  the  difficulty  of  the  change  detection  task.  In  the  generation  of  stimuli 
for  Experiment  II,  38  of  the  4G  samples  of  photography  used  were  originally 
1:20,  000  scale  or  close  to  it  (see  Appendix  III),  and  in  the  1:10,  000  test  material, 
12  samples  of  1:20,000  photography  were  used  and  one  sample  of  1:40,  000. 
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MEAN  TARGET  DETECTIONS  PER  MINUTE  AS  A 
FUNCTION  OF  SCALE  Non-P.l. 


This  would  be  expected  to  result  in  poorer  quality  at  the  largest  scale  than  at 
the  other  two  scales  and  might  account  tor  the  fact  that  change  detection  perform¬ 
ance  at  the  1:20,000  scale  was  better  overall  than  at  the  1:10,000  scale. 

The  problem  of  differences  in  the  number  of  target  changes  also  occurs  when 
background  complexity  is  varied.  There  are  75  target  changes  in  the  industrial- 
background  material,  60  in  the  surburban,  and  44  in  the  rural.  If  complexity 
is  assumed  to  vary  directly  with  the  number  of  target  changes,  the  suburban 
material  is  nearly  equidistant  from  the  industrial  and  rural  backgrounds  on  the 
complexity  continuum.  Again,  Table  XII  shows  that  inspection  time  increases 
with  the  number  of  changes,  but  it  does  not  rise  in  proportion  to  the  changes. 

As  in  the  case  with  the  scale  factor,  the  non-photointerpreters  spend  proportion¬ 
ately  less  time  with  the  photographic  pairs  that  contain  more  changes. 

Referring,  as  before,  to  the  percentage  of  correct  detection  figures  (Table  XII), 
this  performance  measure  bears  no  simple  relationship  to  background  complexity. 
The  number  of  correct  detections  per  unit  time  as  a  dependent  measure  (Figure 
28)  results  in  a  clear  and  direct  relationship  between  performance  and  the  fre¬ 
quency  of  target  change  in  the  stimulus  material.  Furthermore,  in  a  plot  of  the 
detections  per  unit  time  for  the  three  scales  with  background  type  as  the  para¬ 
meter,  the  curves  contrast  markedly  with  those  shown  in  Figure  23,  which  shows 
inspection  time  per  scale  where  the  time  measure  is  not  adjusted  for  the  number 
of  detections.  The  lower  curve  (rural)  rises  sharply  in  this  plot,  whereas  it  is 
essentially  flat  when  the  number  of  detections  per  unit  time  is  the  dependent 
measure  (see  Figure  29).  The  other  two  curves  are  correspondingly  displaced 
by  this  procedure. 

The  decision  as  to  which  performance  measure  is  best  depends,  of  course,  on 
considerations  other  than  those  presented  here.  The  user  of  the  information  sup¬ 
plied  by  this  study  will  decide  whether  it  is  preferable  to  use  one  measure  or 
another.  Either  kind  of  situation  could  occur  —  information  about  a  large  or 
complex  area  is  needed  rapidly  with  relatively  little  concern  about  omissions; 
or,  the  maximum  probability  that  whatever  relevant  changes  there  are  will  be 
detected  by  the  photointerpreter  is  essential.  The  purpose  in  presenting  the 
data  in  more  than  one  way  is  to  extend  its  utility  and  permit  an  easier  application 
to  a  broader  range  of  practical  situations.  This  is  a  task  the  authors  are  better 
able  to  accomplish  than  the  reader  who  acquires  the  report  because  of  a  specific 
and  urgent  need. 

The  question  of  the  effect  of  artificially  introduced  targets  on  performance  was 
raised  in  the  "Conclusions”  discussion  section  for  Experiment  I,  In  Experiment 
II,  artificial  target  changes  were  added  to  those  existing  in  the  original  photo¬ 
graphy.  Table  XVI  shows  the  percentage  of  targets  introduced  by  the  artist  and 
the  correct  detections  made  of  them  as  a  percentage  of  all  correct  detections. 
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TARGET  DETECTIONS  PER  MINUTE 


Figure  28  TARGET  DETECTIONS  PER  MINUTE 
AS  A  FUNCTION  OF  BACKGROUND 
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Figure  29  TARGET  DETECTION  PER  MINUTE  AS  A  FUNCTION  OF 
SCALE  FOR  THREE  TYPES  OF  BACKGROUND 


Table  XVI  FREQUENCY  OF  OCCURRENCE  OF  ARTIFICIAL  TARGETS 
AMONG  REAL  TARGETS  AND  FREQUENCY  OF  DETECTIONS 


DISPLAY  SCALE 

PERCENT  OF 

TARGET  OCCURRENCES 

PERCENT  OF 
DETECTIONS 

ARTIFICIAL 

REAL 

ARTIFICIAL 

REAL 

1:10,  0C0 

(P.l. 

18 

82 

23 

77 

INon-P.I. 

30 

70 

45 

55 

1:20,  000 

Non-P.l. 

32 

68 

34 

66 

<p.i. 

13 

87 

14 

86 

1 :40,  000 

\  Non-P.l. 

17 

83 

17 

83 

The  only  real  difference  between  the  frequency  of  artificial  targets  and  the  pro¬ 
portion  of  correct  detections  attributable  to  them  occurs  in  the  1 :10,  000-scale 


to  produce  the  test  material  (i.o.,  the  artificial  targets  were  loss  blurred  than 
the  inherent  changes  blurred  through  enlargement).  Although  the  experimental 
design  did  not  permit  the  inclusion  of  performance  measures  from  both  observer 
groups  in  a  single  analysis  of  variance,  a  series  of  t -ratios  were  computed  for 
correct  detections  and  inspection  limes  (Table  XV).  These  ratios  revealed  no 
significant  differences  between  the  photointerpreters  and  the  non-photo  interpreters 
for  the  two  measures.  Similarly,  a  median  test  of  the  difference  between  groups 
in  false  detections  did  not  reach  statistical  significance. 


An  inspection  of  the  graphic  data  (see  Figures  24,  25,  and  2G)  leads  one  to  sus¬ 
pect  that  greater  control  over  the  within-subject  or  within-group  sources  of 
variability  would  result  in  the  demonstration  of  statistically  significant  differ¬ 
ences  in  some  instances.  For  example,  there  appears  to  be  a  general  tendency 
for  the  non-photointerpreters  lo  be  slightly  superior  in  target-change  detection 
with  photography  of  1:10,000  scale,  but  markedly  inferior  to  the  photointerpreters 
with  the  1 :40,  000  scale.  The  non-photointerpreters  also  appear  to  take  a  little 
longer  with  the  1:40,000  scale.  Of  the  eight  comparisons  of  errors  of  commis¬ 
sion  shown  in  Figure  20,  only  once  is  the  frequency  greater  for  the  photointer¬ 
preters.  Thus,  in  overall  performance,  the  photo  interpreters  appear  to  be 
superior,  with  this  superiority  more  evident  with  the  smaller  scale  (1:40,000). 

The  data  obtained  under  the  conditions  of  Kxperimenl  II  indicate  that  the  apparent- 
motion  display  is  superior  to  the  side-by-side  display  in  the  lime  expended  by 
non-photointerpreters  and  pholoinlorprcters  to  detect  change  in  comparative- 
cover  photography  (p  less  than  0.05).  There  is  no  statistically  significant  dif¬ 
ference  belween  the  two  display  methods  in  their  effect  on  the  percentage  of 
correct  detections  and  the  frequency  of  false  detections  for  the  analyses  of 
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variance  summarized  in  the  ’’Results"  section  of  Experiment  II.  This  method 
of  analysis  was  selected  at  the  time  the  experimental  design  and  methodology 
were  developed.  Despite  the  compromises  imposed  on  the  methodology  by  the 
limited  stimulus  material,  the  planned  analysis  was  executed.  However,  there 
is  some  cause  to  question  the  desirability  of  using  percentages  based  on  such 
small  numbers  as  those  shown  in  Appendix  V. 

In  answer  to  this,  at  the  cost  of  sacrificing  the  analysis  of  one  factor,  the  scores 
for  background  complexity  were  combined  before  percentages  were  computed. 
The  results  of  this  analysis  for  the  non-photointerpreters  are  summarized  in 
Table  XVII.  Blur  and  scale  were  significant,  as  before,  but  with  larger  F 
ratios.  Moreover,  the  main  effect  of  display  method  shows  the  apparent- motion 
method  to  be  significantly  superior  to  the  side-by-side  method.  The  means  for 
these  treatments  according  to  this  second  analysis  are  shown  in  Table  XVIII. 

Since  all  three  [actors — scale,  blur,  and  method  —  were  statistically  significant 
in  the  second  analysis  (as  opposed  to  just  two  factors,  blur  and  scale,  in  the 
first),  it  appears  desirable  that  further  studies  be  designed  so  that  detection 
scores  under  all  combinations  of  the  treatment  conditions  are  comparable  with¬ 
out  a  percentage  transformation.  This  approach  would  reduce  an  important 
source  of  error  variance,  the  use  of  percent-correct  detection  scores,  which 
are  highly  sensitive  to  minor  performance  differences  when  observations  are 
limited  in  number. 

The  character  of  the  stimulus  material  available  for  this  experiment  imposed 
severe  conditions  for  testing  the  display  methods.  Although  the  same  photo¬ 
graphy  was  used  for  both  displays,  the  effect  was  more  likely  to  be  severe  for 
the  apparent- motion  display  than  for  the  side-by-side  display.  The  reason  for 
this  is  that,  with  the  sensitivity  of  the  apparent- motion  method  to  any  type  of 
change,  a  large  amount  of  irrelevant  change  requires  more  extensive  search 
for  relevant  changes.  When  this  irrelevant  "motion"  exceeds  some  value,  the 
requirements  for  search  and  discrimination  in  the  apparent- motion  display  will 
tend  to  nullify  the  gains  obtainable  with  this  technique  under  conditions  with  less 
noise.  The  photography  used  in  Experiment  II  spanned  a  number  of  years  for 
most  comparative  pairs  and,  in  this  .sense,  may  not  be  typical  of  current  com¬ 
parative-cover  photography  whore  significant  target  changes  occur  in  consider¬ 
ably  shorter  time,  frequently  in  a  matter  of  days. 

Improvements  in  reconnaissance  systems  could,  presumably,  result  in  the 
acquisition  of  comparative-cover  photography,  which  is  more  amenable  to  the 
apparent- mot  ion  method  of  display.  Thus,  in  addition  to  the  development  of 
better  sensors  and  sensor  platforms,  attention  would  be  directed  toward  the 
better  duplication  of  flight  path,  altitude,  ground  speed,  time  of  day,  and  any 
other  factors  that  produce  irrelevant  visual  noise  when  uncontrolled.  It  the 
photointerpretation  function  is  the  bottleneck  in  the  intelligence  system,  efforts 
to  control  these  factors  may  prove  Fruitful. 
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Table  XVII  SUMMARY  OF  ANALYSIS  OF  VARIANCE 
Correct  Detections  With  Background  Complexities  Combined 
Before  Percentage  Transformations 


NON-P.l.  GROUP 


SOURCE  OF  VARIATION 

61 

ms 

F 

Between  Subject  Total 

(17) 

Scales  (S) 

2 

12365 

13.00** 

Error  (Between  S's  Within  Groups) 

15 

951 

Within  Subject  Total 

(90) 

Methods  (M) 

1 

1281 

4.372* 

Blurs  (B) 

2 

3006 

10.259** 

M  x  S 

2 

141 

M  x  B 

2 

202 

S  x  B 

4 

285 

M  x  S  x  B 

4 

160 

Within  Subject  Error 

75 

293 

TOTAL 

107 

* 
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Significant  at  0.05  Level 
Significant  at  0.01  Level 


Table  XVIII  MEAN  PERCENT  DETECTION  SCORES  FOR  NON-P.I. 
Terrain  Complexities  Combined 


SCALE  1:10,000 


IMAGE 

QUALITY 

AM 

sxs 

M 

Hi 

73.5 

62.7 

Med 

56.3 

46.8 

Lo 

37.2 

45.5 

M 

56.8 

52.0 

54.4 

SCALE  1:20,000 

!MAr;E 

QUALITY 

AM 

SXS 

M 

Hi 

76.2 

60.8 

68.5 

Med 

65.2 

60.3 

62.8 

Lo 

64.0 

49.8 

56.9 

M 

68.4 

57.0 

62,7 

SCALE  1:40,000 

IMAGE 

QUALITY 

AM 

SXS 

M 

Hi 

38.3 

31.5 

34.9 

Med 

28.5 

23.2 

25.7 

Lo 

21.7 

20.5 

21.1 

M 

29.5 

25.1 

27.3 

G.M. 

51.6 

44.7 

48.1 
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EXPERIMENT  III 


INTRODUCTION 

The  results  of  previous  research  would  seem  to  indicate  that  the  greater  the 
amount  of  "visual  noise"  or  irrelevant  change  appearing  between  the  two  samples 
of  any  comparative  pair  of  photographs,  the  less  effective  is  the  apparent-motion 
display  method  in  facilitating  rapid,  accurate,  and  complete  extraction  of  intel¬ 
ligence.  In  an  attempt  to  explore  this  stimulus  parameter  more  systematically, 
a  third  experiment  was  conducted  in  which  the  time  lapse  between  comparative 
coverages  was  systematically  varied  and  target-detection  performance  by  photo- 
interpreters  and  non-photointerpreters  was  assessed  using  both  side-by-side  and 
apparent-motion  display  systems.  If  the  assumption  is  accepted  that  with  longer 
time  lapses  between  successive  coverages  there  generally  will  be  more  numerous 
irrelevant  differences  (those  not  critical  to  successful  completion  of  the  military 
mission)  occurring  between  the  two  photographs,  a  prediction  would  be  that 
interpretation  would  take  longer  and  be  less  accurate  and  complete  for  longer 
time-base  materials.  It  might  further  be  hypothesized  that,  because  of  the 
enhancement  and  attention -demanding  nature  of  differences  between  the  two 
photographs  with  the  apparent-motion  display  method,  such  interference  might 
be  particularly  deleterious  to  inspection  performance  with  this  display  system. 

SUBJECTS 


Two  groups  of  12  subjects  each  served  as  observers  in  this  experiment.  One 
group  (non -photointerpreters)  was  composed  of  shop  and  laboratory  technicians 
and  professional  personnel  from  within  the  Bioastronautics  organization  of  The 
Boeing  Company.  These  observers  had  neither  formal  training  nor  extensive 
experience  in  photointerpretation,  although  eight  had  also  participated  in  Experi¬ 
ment  II.  The  second  group  consisted  of  12  Boeing  employees  who  were  trained 
or  experienced  professional  photo  interpreters.  No  member  of  this  group  was 
currently  active  in  photointerpretation  as  a  regular  part  of  his  job.  Of  these  12 
observers,  two  had  participated  only  in  Experiment  IT,  one  had  served  only  in 
Experiment  I,  and  five  had  served  as  observers  in  both  Experiments  I  and  II. 

APPARATUS 

The  display  equipment  used  to  present  the  comparative  pairs  in  both  side-by- 
side  and  apparent-motion  formats  was  the  same  as  that  employed  in  the  two 
previous  studies.  All  optics  were  cleaned,  and  the  light  sources  were  replaced 
and  calibrated  before  the  first  data  were  collected. 

The  response  system  was  the  same  as  that  described  earlier.  Since  only  a 
single  target  class  (antiaircraft  artillery)  was  designated  for  detection  and 
reporting,  that  portion  of  the  display  and  response  systems  introduced  pre¬ 
viously  for  purposes  of  target  identification  was  not  utilized  in  the  present  study. 
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STIMULUS  MATERIAL 


Four  vertical  photographic  coverages  of  a  3.6-mile-long  and  2. 2 -mile -wide 
strip  of  terrain  along  the  east  shore  of  Lake  Washington  near  Seattle,  Wash¬ 
ington,  was  purchased  from  a  local  aerial  survey  service.  The  original  photog¬ 
raphy  was  taken  from  an  altitude  of  approximately  12,000  feet  with  a  camera 
having  a  6 -inch  focal  length.  The  working  prints  were  enlargements  to  approxi¬ 
mately  a  1:7,500  scale,  resulting  in  an  18-  by  30-inch  format  size.  The  earliest 
of  the  four  coverages  (A)  was  taken  on  23  Fabruary  1962  and  the  second  coverage 
(B)  was  taken  on  25  July  1962.  The  third  picture  (C)  was  taken  on  24  December 
1962  and  the  most  recent  coverage  (D)  was  taken  on  17  May  1963.  Thus,  com¬ 
parative  pairs  consisting  of  samples  A-B,  A-C,  and  A-D  represented  time 
lapses  of  147,  292,  and  461  days,  respectively,  or  time  bases  of  approximately 
5,  10,  and  15  months.  These  four  coverages  are  shown  at  reduced  scale  in 
Figures  30,  31,  32,  and  33. 

The  dominent  terrain  features  were  those  typically  classified  as  suburban. 

Ground  activities  observed  during  the  15 -month  period  sampled  ranged  from 
the  construction  of  light  manufacturing  and  production  complexes  to  relatively 
unchanged  rural  areas.  The  most  substantial  portion  of  the  photography  de¬ 
picted  the  rapid  development  of  suburban  housing  projects.  The  original  photog¬ 
raphy  was  of  good  quality  and  only  a  slight  degradation  resulted  from  the  3.2x 
enlargement. 

From  the  18-  by  30-inch  working  prints  representing  each  coverage  (A,  B,  C, 
and  D),  12  comparable  4-  by  4- inch  sections  were  selected.  Four  of  these 
sections  are  shown  in  Figure  34.  Three  antiaircraft  artillery  installations 
were  artistically  introduced  within  each  4-  by  4-inch  area  on  coverages 
B,  C,  and  D.  The  schematic  shown  in  Figure  35  was  used  as  a  model  for  the 
drawing  of  the  emplacements.  The  location  of  the  artillery  batteries  within  any 
given  segment  of  a  photograph  was  the  same  for  coverages  B,  C,  and  D  (the 
comparison  photographs);  but  none  of  the  emplacements  incorporated  on  coverage 
A  (the  standard)  were  coincident  with  any  of  those  appearing  in  the  three  com¬ 
parison  samples. 

Negatives  of  each  of  the  48  areas  (12  areas  on  each  of  the  four  coverages)  were 
prepared,  and  positive  transparencies  were  made  from  them.  A  minimal  amount 
of  rectification  and  scaling  (involving  only  17  of  the  48  negatives)  was  performed 
in  the  photographing  process  to  compensate  for  slight  differences  in  aircraft 
altitude  and  positional  differences  relative  to  the  nadir  on  the  original  photography. 

Each  transparency  was  then  sandwiched  between  two  machined  plates  of  0. 125- 
inch  plexiglass  lor  aecurage  insertion  and  positioning  in  the  display  system. 
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Figure  30  EARLIEST  PHOTOGRAPHIC  SAMPLE  FOR  EXPER1MEN 
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Figure  31  FIVE-MONTH  SAMPLE  FOR  EXPERIMENT  III 
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Figure  33  FIFTEEN-MONTH  SAMPLE  FOR  EXPERIMENT  III 
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Figure  35  SCHEMATIC  REPRESENTATION  OF 
A  TYPICAL  ANTIAIRCRAFT  EMPLACEMENT 


TRAINING 


Those  subjects  who  had  not  previously  been  trained  in  conjunction  with  their 
participation  in  the  earlier  experiments  were  given  the  training  procedure  des¬ 
cribed  in  detail  for  Experiment  I.  Subjects  having  received  previous  training 
and  experience  with  the  display  apparatus  and  response  system  were  given  a 
brief  review  and  refamiliarization  with  the  display  system  and  use  of  the  res¬ 
ponse  panel.  Three  sets  of  the  "high  visual  noise  aerial  photographs"  were 
presented  and  knowledge  of  performance  provided. 

INSTRUCTIONS 

The  following  instructions  were  presented  to  each  observer  immediately  before 
beginning  the  data  collection: 

The  specific  targets  you  will  be  asked  to  search  for  in  the  aerial  photo¬ 
graphs  to  follow  are  ANTIAIRCRAFT  ARTILLERY.  Please  study  this 
schematic  of  a  typical  antiaircraft  emplacement  while  1  describe  some 
of  the  identifying  characteristics  of  such  an  installation. 

The  term  ANTIAIRCRAFT  ARTILLERY  is  applied  to  ground  and  ship- 
borne  weapons  and  materials  used  to  locate,  fire  on,  and  destroy 
enemy  aircraft. 

Antiaircraft  artillery  batteries  generally  give  the  interpreter  little 
difficulty  because  of  their  characteristic  layout  when  emplaced  for 
firing.  The  normal  battery  will  consist  of  six  guns,  each  found  in  a 
circular  revetment,  distributed  around  fire-control  equipment  which 
may  include  visual  height- range  finders,  gun  directors,  radar  and 
power  plants. 

The  COMMAND  POST  may  be  sited  either  in  the  center  or  behind 
the  guns,  but  usually  within  150  feet.  The  command  post  is  the  con¬ 
trol  center  for  the  battery  and  can  be  found  in  a  van-type  vehicle  or 
in  earthen  or  concrete  bunkers. 

COMMUNICATIONS  CABLES  are  buried  in  trenches  which  run  from 
the  command  post  to  the  gun  area  and  then  split  into  various  cables 
leading  to  eacli  gun. 

The  POWER  for  this  system  may  be  supplied  by  either  generators 
or  batteries.  Power  cables  may  be  underground  or  overhead,  sup¬ 
ported  by  poles. 

Permanent  pathways  are  sometimes  laid  down  in  the  case  of  fixed 
batteries  and  usually  show  up  with  a  light  tone  when  compared  with 
the  surrounding  country. 

Since  the  main  function  of  antiaircraft  artillery  is  to  protect  vital 
ground  targets  from  attack  by  aircraft,  the  guns  are  usually  located 
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on  hilltops,  in  open  fields,  lots  or  parks,  or  on  top  of  buildings  to 
allow  a  360 -degree  field  of  fire.  Camouflage  is  not  normally  used 
but  emplacements  may  be  partially  camouflaged  on  the  sides. 

It  will  be  your  task  to  detect  and  report  all  instances  where  antiair¬ 
craft  batteries  have  either  been  a.v  1  or  removed  from  the  area  being 
searched.  Work  as  fast  as  you  ca  ;„»ut  be  sure  you  have  found  all  the 
targets  before  signalling  completion  Speed  and  accuracy  are  weighted 
equally  in  the  evaluation  of  your  pf*  -  *mance. 

EXPERIMEN  ■  DESIGN 

The  basic  experimental  design  is  pi*  *  thematically  in  Figure  36.  Each 
photointerpreter  inspected  all  twelve  ox  x-  by  4-inch  sectors  comprising  the 
total  area.  Six  of  these  were  viewed  side  by  side  and  six  were  seen  in  apparent 
motion.  Of  the  six  areas  searched  by  each  photointerpreter  under  either  dis¬ 
play  system,  two  represented  the  5-month  time  base,  two  represented  the  10- 
month  time  base,  and  two  represented  the  16-month  time  base.  Because  there 
were  12  independent  area  samples  available,  it  was  possible  to  obtain  two  per¬ 
formance  samples  from  each  photointerpreter  under  each  of  the  display  method 
times  time-base  conditions  without  viewing  each  area  more  than  once. 

The  display  methods  were  alternated  after  every  two  presentations.  Display- 
method  order  was  counterbalanced  between  subjects  within  each  group.  Thus, 
the  first  subject  would  inspect  the  first  two  comparative  pairs  in  side-by-side 
presentation  and  the  next  two  comparative  pairs  in  apparent  motion  while  the 
second  subject  would  see  the  first  two  pairs  in  apparent  motion  and  the  next 
two  side  by  side. 

To  prevent  any  systematic  biasing  of  the  data  because  of  the  presentation  order 
oi  the  three  different  time  bases,  a  Latin  square  design  was  applied  in  assigning 
time-base  sequence  within  and  between  subjects  in  each  group. 

The  assignment  of  specific  coverage  areas  (from  within  the  12  sections  com¬ 
prising  the  standard  photograph,  A)  to  subject  times  display  method  times 
time -base  conditions  was  random,  with  the  restriction  that  each  area  was  to  be 
inspected  by  only  two  subjects  within  each  group  and  under  each  of  the  six  dis¬ 
play  method  times  time-base  conditions. 

RESULTS 

The  performance  measures  obtained  from  each  observer  were  the  total  time 
required  to  inspect  each  comparative  pair  and  the  number  of  detections  reported 
for  each.  Since  ground  truth  (the  actual  number  of  targets  in  the  area)  was 
known,  it  was  possible  to  determine  completeness  (percent  of  targets  detected) 
as  well  as  the  number  of  errors  of  commission.  Table  XIX  summarizes  the 
percentage  of  correct  detections,  the  mean  time  per  comparison,  and  the  mean 
number  of  errors  of  commission. 
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Figure  36  SCHEMATIC  REPRESENTATION  OF  DESIGN  FOR  EXPERIMENT  III 


Table  XiX  MEANS  OF  PERFORMANCE  MEASURES 
FOR  EXPERIMENT  111 


(A)  Percent  Detections 


5-MONTH  BASE  10-MONTH  BASE  15-MONTH  BASE 
AM  SxS  AM  SxS  AM  SxS 

M 

Non-P.l.  Group 

33.3  J  42.7 

i 

39.6  j  36.5 
_ 1 _ 

28.1  j  39.6 

_ i _ 

36.6 

P.i.  Group 

38.5  J  39.6 

i 

BQHE9I 

35.9 

M  = 

38.5 

36.5 

33.8 

(B)  Mean  Time  Per  Pair  (Seconds) 

5-MONTH  BASE 
AM  SxS 

10-MONTH  BASE 
AM  SxS 

15-MONTH  BASE 
AM  SxS 

M 

Non-P.l*  Group 

94.7  {  101.0 

_ I _ 

94.7 

P.i.  Group 

99.2 

M  = 

95.5 

99.3 

96.1 

(C)  Mean  Errors  of  Commission 

Per  Pair 

5-MONTH  BASE 
AM  SxS 

10-MONTH  BASE 
AM  SxS 

15-MONTH  BASE 
AM  SxS 

M 

Non-P.l.  Group 

.215 

P.I.  Group 

0.333  J  0.375 

1 

.382 

M  = 

0.250 

0.250 

0.396 

DISPLAY  METHOD 

Percent  Detections 
Mean  Time  (Seconds) 

Commission  Errors 


AM _ SxS 


35.40 

37.2 

92.8 

101.1 

.271 

.326 
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The  analysis  of  variance  based  on  the  number  of  correct  detections  (Table  XX) 
reveals  no  significant  differences  either  as  a  function  of  presentation  method 
(35.4  percent  with  apparent  motion  as  compared  to  37.2  percent  with  side  by 
side),  level  of  experience  of  the  subjects  (3G.G  percent  for  non-photointerpre¬ 
ters  and  35.  9  percent  for  photointerpreters),  or  the  time  base  between  compar¬ 
ative  samples  (38.5,  36.  5,  and  33.8  percent  for  time  bases  of  5,  10,  and  15 
months,  respectively). 


Table  XX  SUMMARY  OF  ANALYSIS  OF  VARIANCE  OF 
CORRECT  DETECTIONS 


SOURCE  OF  VARIATION 

df 

ms 

F 

Between  Groups  (G) 

1 

0.11 

— 

Between  Display  Methods  (D) 

1 

0.69 

— 

Between  Time  Bases  (T) 

2 

1.70 

— 

G  x  D 

1 

4.00 

1.31 

G  x  T 

2 

0.36 

D  x  T 

2 

3.70 

G  x  D  x  T 

2 

0.09 

■El 

Error  Variance 

132 

3.05 

■9 

TOTAL 

143 

Similarly,  the  analysis  of  inspection  time  required  to  search  each  comparative 
pair  (Table  XXI)  revealed  no  significant  differences  cither  as  a  function  of  the 
display  method  involved  (92.8  seconds  with  apparent  motion,  as  compared  with 
1  01 . 1  seconds  with  the  side-by-side  display),  or  the  experience  level  of  the 
observer  (94.7  seconds  required  by  non-pholointerprolers  and  99.2  seconds 
required  for  photointerpreters),  or  the  time  lapse  between  coverages  (95.5,  99.3, 
and  9G.1  seconds  for  the  5-,  10-,  and  15-month  separations,  respectively). 

The  mean  number  of  commission-error  responses  made  by  the  pholointcrpre- 
ters  (0.382  per  comparative  pair)  was  significantly  greater  (p  less  than  0.05) 
than  the  mean  number  committed  by  the  non-photoinlerprelers  (0.215).  See 
Table  XXII  i'or  the  summary  of  the  analysis  of  variance.  The  rale  at  which 
commission  errors  were  made  was  not,  however,  related  to  the  display  method 
(0.271  lor  apparent  motion  and  0.32(5  for  side  by  side)  or  to  the  time  base  of 
the  photography  (0.250,  0.250,  and  0.39G  for  time  bases  of  5,  10,  and  15 
months,  respectively)  (See  also  Figures  37  and  38). 


109 


SUMMARY  OF 


Table  XXI 

ANALYSIS  OP  VARIANCE  OP  INSPECTION  TIMES 


SOURCE  OF  VARIATION 

df 

ms 

F 

Between  Groups  (G) 

1 

0.81 

Between  Display  Methods  (D) 

1 

2.72 

1.62  N.S. 

Between  Time  Bases  (T) 

2 

0.22 

— 

G  x  D 

1 

1.32 

— 

G  x  T 

2 

0.03 

— 

D  x  T 

2 

0.17 

— 

G  x  D  x  T 

2 

0.21 

_ 

Error  Variance 

132 

1.68 

— 

TOTAL 

143 

Table  XXII  SUMMARY  OE 

ANALYSIS  OE  VARIANCE  OE  ERRORS  OP  COMMISSION 


SOURCE  OF  VARIATION 

df 

ms 

F 

Between  Groups  (G) 

i 

i 

4.0 

4.55* 

Between  Display  Methods  (D) 

1 

0.44 

_ 

Between  Time  Bases  (G) 

2 

1  .36 

1.54 

G  x  D 

1 

0.03 

— 

G  x  T 

2 

0.09 

— 

D  x  T 

2 

0.20 

— 

G  x  D  x  T 

2 

0.11 

_ 

Error  Variance 

132 

0.88 

— 

TOTAL 

143 

*  Significant  beyond  0.05  level. 
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EAN  TIME  PER  PAIR  (SECONDS) 


TIME  LAPSE  (MONTHS) 


Figure  37  PERFORMANCE  PLOTTED  AS  A  FUNCTION  OF  TIME  LAPSE 


CONCLUSIONS 


The  major  purpose  of  Experiment  III  was  to  compare  the  effects  of  the  appar¬ 
ent-motion  and  side-bv-side  display  methods  upon  target-detection  performance 
where  the  comparative  coverages  spanned  15  months  in  three  equal  intervals. 

As  in  the  previous  experiments,  both  experienced  and  inexperienced  photoin¬ 
terpreters  were  tested. 


The  statistical  analysis  of  the  empirical  data  suggests:  (1)  that  neither  display 
method  is  superior  to  the  other,  (2)  that  target-change  detection  performance 
does  not  vary  for  time  bases  ranging  from  5  to  15  months,  and  (3)  that  non- 
photointerpreters  make  fewer  false  detections  than  photoinlerpreters,  but  that 
these  two  groups  do  not  differ  in  the  number  of  correct  detections  or  inspec¬ 
tion  time. 

The  question  of  primary  concern  is  the  extent  to  which  the  observed  results  may 
be  applied  to  the  phntointerpretation  task  in  general.  This  involves  a  critical 
data  appraisal  for  the  purpose  of  examining  alternate  hypotheses  which  may 
account  for  the  obtained  results.  Only  one  statistically  significant  difference 
was  apparent  in  the  analysis  of  the  data:  this  was  a  difference  between  groups 
in  1  he  number  of  false  detection,  the  experienced  group  yielding  a  significantly 
larger  number  of  commission  errors.  This  difference  may  represent  a  tendency 
for  experienced  observers  to  attach  more  meaning  to  observed  changes  of  all 
kinds,  including  irrelevant  ones. 

Only  one  type  of  target  was  introduced  into  the  photography  in  Experiment  III. 
Although  the  search  for  a  single  type  of  target  is  not  unrealistic  in  tactical 
operations,  it  imposes  certain  limitations  on  the  interpretation  of  the  data 
secured  in  the  laboratory  investigation  of  the  two  display  methods.  Specifically, 
the  stimulus  conditions  permitted  no  distinction  between  the  detection  of  a  tar¬ 
get  change  and  its  identification.  If  the  observer  saw  an  antiaircraft  battery 
and  reported  it,  the  response  was  recorded  as  a  correct  detection.  In  die  side- 
bv-side  display  condition,  the  observer  could  have  decided,  after  several  corn- 
paid  sons  of  the  two  members  of  each  pair,  that  no  comparison  was  necessary, 
that  the  detection  of  an  emplacement  was  sufficient  without  checking  the  second 
coverage  to  determine  whether  it  was  also  present  there.  Any  battery  detected 
may  have  been  assumed  by  the  subject  to  have  always  been  either  added  or  re¬ 
moved.  This  situation  would  tend  to  favor  the  side-by-side  method  over  that 
which  obtained  in  previous  experiments,  where  a  change  might  be  one  of  size  or 
position  and  where  a  variety  of  targets  was  included.  Moreover,  the  targets 
were  relatively  small  in  area  and  typically  low  in  contrast  with  the  background, 
as  can  be  seen  in  Figure  :M.  The  advantage  that  the  apparent-motion  method 
might  have  retained  over  the  side-by-side  display  may  have  been  lost  because 
of  this.  Figure  11(A),  in  the  "Results”  section  of  Experiment  I,  shows  that 
detection  scores  for  the  two  display  methods  become  less  differentiated  at  the 
lower  contrast  levels. 


In  addition  lo  nonlarget  man-made  changes,  there  were  many  others  involving 
sun  angle,  overall  brightness  and  contrast,  seasonal  variations,  and  variations 
in  the  quality  of  the  original  photography.  These  occur,  of  course,  in  opera¬ 
tional  photography:  but  the  unique  sample  used  in  Experiment  III  does  not  per¬ 
mit  any  evaluation  of  this  source  of  variability.  The  possibility  exists  that  the 
particular  sample  used  was  not  representative  of  the  population. 

The  foregoing  is  not  an  attempt  to  "explain  away"  the  empirical  results,  but 
rather  represents  a  necessary  part  of  the  critical  evaluation  of  any  piece  of 
research.  It  is  included  here  in  the  belief  that  no  report  is  complete  that  does 
not  bring  the  experimenter's  intimate  knowledge  of  research  to  bear  on  its 
criticism.  The  limitations  listed  are  the  usual  ones  associated  with  experi¬ 
mentation  of  necessarily  limited  scope;  they  are  useful  for  the  design  of  better 
subsequent  research. 
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GENERAL  CONCLUSIONS 


The  three  experiments  completed  under  this  contract  have  contributed  a  sub- 
stantial  amount  of  new  data  on  change  detection  from  comparative-cover  aerial 
photography.  However,  there  are  certain  limitations  that  affect  the  interexperi¬ 
ment  comparisons  and  generalizations  of  conditions  deviating  from  those  under 
which  the  data  were  obtained.  The  major  restrictions  stem  from  two  primary 
sources:  (L)  the  characteristics  of  the  stimulus  material,  and  (2)  the  task 
requirements. 

A  very  serious  difficulty,  apparently  inherent  to  this  area  of  investigation,  is 
that  of  obtaining  reconnaissance  photography  that  (1)  allows  precise  control  over 
those  variables  under  investigation  while  holding  all  others  constant,  and  (2)  is 
representative  of  operational  imagery  to  the  extent  that  the  data  are  maximally 
and  immediately  applicable.  The  research  reported  here  is  no  less  subject  to 
these  difficulties;  although,  among  the  three  studies,  various  degrees  of  control 
over  pictorial  content  and  representativeness  are  included.  A  brief  review  of 
the  major  characteristics  and  differences  in  the  stimulus  material  employed  in 
the  three  experiments  follows. 

Kxperimenl  I  —  This  material  can  be  classified  as  "Homogeneous  Noise  — 
Heterogeneous  Targets."  The  homogenily  in  noise  results  from  the  procedure 
involved  in  obtaining  the  comparative  pairs.  Since  comparative  pairs  were 
obtained  from  successive  photographs  taken  during  the  same  reconnaissance 
mission,  visual  noise  (irrelevant  changes)  was  restricted  to  one  type.  The 
homogeneity  of  visual  noise  stems  almost  exclusively  from  the  difference  in 
"taking"  angle  associated  with  an  invariant  lime  base.  The  heterogeneity  of 
targets  was  achieved  by  systematically  introducing  a  wide  variety  of  target 
objects  and  changes  of  known  number  and  types, 

Kxperimenl  II  —  The  stimulus  material  used  in  this  experiment  can  be  classified 
as  "Heterogeneous  Noise —  Heterogeneous  Targets.  "  The  large  amount  and 
variety  of  noise  arose  from  the  inclusion  of  photography  representing  wide  and 
variable  lime  buses,  without  controlling  the  time  of  day,  season  of  the  year, 
atmospheric  conditions,  photographic  equipment,  or  position  of  the  reconnais¬ 
sance  vehicle.  Since  changes,  both  relevant  and  irrelevant,  were  numerous 
and  "ground  truth"  was  not  known,  a  variety  of  target  classes  was  specified  for 
detection  and  a  group  concensus  criterion  was  employed  in  the  evaluation.  A 
small  proportion  of  artistically  introduced  targets  were  also  included  to  provide 
data  for  evaluating  this  procedure  to  establish  "ground  trulir'  for  experimental 
imagery. 


Kxperimenl  III  —  The  material  involved  in  this  study  is  classified  as  "Hetero¬ 
geneous  Noise —  Homogeneous  Targets.  "  Tins  imagery,  procured  locally,  was 
originally  obtained  for  real-estate  evaluation.  Thus,  time  base,  time  of  day, 
camera  equipment,  and  spatial  location  o!  the  aircraft  were  "controlled." 
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Foliage  and  ground-cover  changes  associated  with  seasonal  differences  and  man¬ 
made  changes  not  of  military  significance  contributed  a  moderate  amount  of  hetero¬ 
geneous  visual  noise.  The  specific  targets  (antiaircraft  artillery  batteries) 
designated  for  detection  were  artistically  introduced  on  the  photography. 

The  task  requirements  also  varied  from  one  study  to  the  next,  thereby  further 
complicating  the  interexperiment  comparisons.  The  nature  of  the  task  was 
most  often  predicated  upon  the  number  and  specific  characteristics  of  the  stimu¬ 
lus  material  available.  In  Experiment  I  the  problem  confronting  the  observer  was 
to  report  each  change  detected  and  to  identify  each  detected  change  by  type  and 
class  of  object.  Since  reporting  all  of  the  large  number  of  changes  represented 
on  the  extended  time-base  imagery  of  Experiment  II  would  have  constituted  a 
prohibitively  lengthy  assignment,  a  specific  target  class  (or  a  few  classes)  was 
designated  as  "relevant"  for  each  comparative  pair.  Generally,  these  target 
classes  were  gross  (i.  e. ,  warehouses,  roads,  harbor  activity,  etc.),  and  each 
photograph  contained  several  examples  of  each  specified  class.  In  Experiment 
III  only  one  specific  and  highly  distinctive  target  was  being  sought,  and  the 
observers  were  thoroughly  familiarized  with  the  appearance  of  this  particular 
target  prior  to  the  search. 

The  significant  findings  relating  to  each  of  the  independent  variables  under  in¬ 
vestigation  are  summarized  by  each  dependent  variable  in  Table  XXIII.  Only 
two  of  these  variables  are  common  to  all  three  experiments  —  display  methods 
and  experience  levels.  Of  the  eleven  comparisons  made  between  the  side-by- 
side  and  apparent -motion  display  methods,  four  revealed  significantly  superior 
performance  with  the  apparent-motion  display,  one  revealed  a  significant  differ¬ 
ence  favoring  the  side-by-side  display,  and  performance  scores  for  the  remainder 
of  the  comparisons  were  essentially  comparable  with  either  display  method. 

Prior  training  or  experience  in  photointerpretation  did  not  appear  to  facilitate 
information-extraction  performance  since,  of  the  eleven  comparisons  made  be¬ 
tween  groups,  only  two  differences  were  significant,  and  both  of  these  favored 
the  non- photo  interpreters. 

The  remaining  independent  variables  listed  in  Table  XXIII  —  contrast,  scale, 
image  quality,  background,  and  time  base  —  were  treated  in  only  a  single 
experiment  and  thus  should  be  interpreted  in  Light  of  the  conditions  existing 
during  the  data  collection.  There  does,  however,  appear  to  be  sufficient  evi¬ 
dence  to  demonstrate  that  as  imagery  is  impoverished,  either  by  reducing  contrast 
or  imposing  blur,  the  proportion  of  targets  detected  will  diminish.  Photographic 
scale  and  background  complexity,  although  significantly  affecting  detection  limes, 
are  not  unambiguous  in  their  relationship  to  total  performance.  Performance 
differences  attributable  to  the  time  base  systematically  investigated  in  Experi¬ 
ment  III  were  not  demonstrable. 
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RECOMMENDATIONS 


In  ton  of  the  olovon  comparisons  of  display  methods  shown  in  Table  XXIII,  per¬ 
formance  with  the  apparent -motion  presentation  was  equal  to  or  better  than  that 
obtained  with  the  side-by-side  display.  There  is  also  substantial  information 
indicating  that  with  certain  types  of  imagery  significantly  more  target  changes 
can  be  detected  and  classified  in  less  time  with  the  apparent-motion  display 
method.  Additional  research  is  needed  to  permit  a  more  precise  specification 
of  the  types  of  reconnaissance  imagery  for  which  the  apparent-motion  method 
is  most  appropriate.  From  the  data  reported  in  this  document  and  that  obtained 
prior  to  this  program,  it  appears  that  the  advantages  of  the  apparent -motion  dis¬ 
play  method  diminish  as  irrelevant  random  visual  noise  increases.  However, 
there  is  evidence  that  such  a  display  system  can  tolerate  a  considerable  amount 
of  systematic  noise  without  an  appreciable  loss  in  effectiveness.  Future  research 
in  this  area  should  attempt  to  use  samples  of  operational  reconnaissance  photo¬ 
graphy  obtained  specifically  for  comparative  cover  analyses.  Mission  require¬ 
ments  could  then  be  established  for  the  acquisition  of  imagery  that  is  within  the 
tolerance  limits  of  such  a  display  system.  Future  investigation  should  also 
include  an  assessment  of  the  use  of  the  apparent -motion  display  for  inspecting 
imagery  other  than  vertical  photography.  For  example,  is  the  display  system 
amenable  to  comparisons  of  oblique  photographs,  of  radar,  or  of  infrared 
imagery?  Can  multisensor  imagery  be  combined  in  such  a  display  system  to 
further  facilitate  information  extraction? 

'Pile  continued  investigation  of  imaging  characteristics  (i.c.,  contrast,  resolution, 
scale,  etc.)  intrinsic  to  various  sensing  systems  and  the  establishment  of  trade¬ 
off  functions  between  these  factors  as  predictors  of  performance  are  highly 
recommended. 

The  display  apparatus  employed  in  the  present  series  of  experiments  can  best 
be  described  as  a  second-stage  prototype.  Based  on  several  hundred  hours  of 
usage,  the  following  recommendations  for  the  construction  of  an  improved  model 
are  offered: 

1)  The  inclusion  of  a  manual  or  automatic  three-directional  staging  unit  for 
aligning  the  two  imagery  samples  being  compared; 

2)  The  design  and  incorporation  of  an  optical-mechanical  rectification  and 
sizing  component: 

2)  The  addition  of  precision  controls  for  varying  alternation  rale  and  independent 
channel  illumination: 

1)  The  provision  for  front  lighting  to  view  opaque  materials  as  well  as  to 
l ransilluminate  positive  and  negative  transparencies; 

5)  The  expansion  of  the  material-handling  capability  to  include  various  formal 
sizes  and  continuous  strip  imagery. 
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These  changes  would  significantly  increase  Ihe  versatility  of  the  display  system 
and  would  permit  the  investigation  of  important  variables  currently  beyond  the 
capabilities  of  the  existing  equipment. 

There  are  also  numerous  procedural  variables  (e.g. ,  the  selection  and  training 
of  observers,  the  amount  of  preintelligence  provided,  the  task  duration  and  work 
load,  and  the  specific  task  requirements)  that  must  be  investigated  for  a  complete 
evaluation  of  the  apparent-motion  display  system  as  a  technique  for  enhancing 
the  speed,  accuracy,  and  completeness  of  target -change  detection  performance. 
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APPENDIX  I 


TRAINING  AND  EXPERIENCE  OF  PHOTO  INTERPRETER 
USED  IN  EXPERIMENTS  I,  II,  AND  III 


Subject 


Training  and  Experience 


TLA 


KAA 


TAB 


HOC 


GED 


Cl  IF 


Completed  Air  Force  photointerpretation  course,  14  weeks,  taken 
in  Orlando.  Florida,  in  1944.  Served  with  Air  Force  Photo 
Mapping  Squadron  prior  to  1946;  worked  at  the  Aero  Charting  Ser¬ 
vice  in  St.  Louis  for  2  months  in  1950;  served  2.5  years  with  the 
67th,  363rd,  and  2nd  Reconnaissance  Technical  Squadrons  from 
1950  to  1953.  where  he  did  photointerpretation  work  in  both  Korea 
and  the  U.S.  Was  in  charge  of  a  Flak  Interpretation  Unit  in  Korea 
and  did  radar  strike  photo  grading  on  practice  missions  and  USAF 
bombing  competitions  (1952).  Has  also  had  experience  in  industrial 
target  analysis,  bomb-damage  assessment  on  foreign  targets,  and 
in  the  reduction  of  map  reference  files  to  color  microfilm  for 
mobility  at  Barksdale  AFB. 

Completed  the  6-week  photinterpretation  course  in  conjunction  with 
the  17-week  Officer's  Intelligence  Course  conducted  by  the  Army 
at  Ft.  Riley,  Kansas,  in  1949.  Utilized  this  training  for  3  years 
as  Battalion  Intelligence  Officer,  Intelligence  and  Reconnaissance 
Platoon  Leader,  and  Regimental  Intelligence  Officer.  Has  had 
experience  in  all  phases  of  conventional  photo  interpretation  methods 
and  procedures  and  for  updating  maps  for  tactical  missions. 

Received  training  in  aerial  photography  and  mapping  at  Colorado 
A&M  College  (14-wcek  R.  O.  T.  C.  course)  and  with  the  Soil  Con¬ 
servation  Service  (6  weeks  plus  field  training).  Worked  3  years 
with  the  Soil  Conservation  Service  in  Colorado  where,  working 
entirely  from  aerial  photographs,  he  was  involved  with  the  deter¬ 
mination  and  measurement  of  soil  conservation  practices. 

Completed  46  weeks  of  training  at  University  of  Washington  from 
1954  to  1956  in  forestry-oriented  photointerpretation.  Courses 
included  photogrammetry .  timber  cruising,  and  road  location. 
Experience  includes  both  private  business  and  U.S.  Forest  Service 
work,  which  utilized  photo  interpretation  for  forest  inventory, 
logging  settings,  road  locations,  fire  control,  and  [jest  control. 

Completed  6  weeks  course.  Pholointerprolalion  for  Air-Dropping 
Cargo,  at  Olmstead  AFB  in  1952.  Thereafter  worked  for  6  years 
for  USAF  relating  aerial  photographs  to  air  drops. 

Received  U.S.  Army  topographical  training  courses  of  unspecified 
extent.  From  September  1939  to  March  1942  duties  as  member  of 
the  photogrammetry  company  of  the  29th  Engineering  Topographical 
Battalion  included  use  of  photo-mapping  equipment.  From  June 
1942  to  March  1944  supervised  similar  work  in  3()th  and  660th 
Engineering  Topographic  Battalions. 
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FSII 


KJii 

GU 

HTK 

KWO 


Training  and  Experience 

Completed  6 -month  photo  mapping  and  interpretation  course  at 
Fort  Beivoir  in  1952.  Received  further  job  training  with  111th 
Reconnaissance  Technical  Squadron.  Fairchild  AFB.  Balance  oi 
USAF  experience  consisted  of  radar  and  photointerpretation, 
mapping,  and  three-dimensional  model  derivation  utilizing  B36 
Tri-mel  film. 

13  weeks  of  schooling  at  Fort  Holabird.  followed  by  5  years  of 
active  reserve  service  in  Detachment  11,  5262nd  AUSU  with  annual 
2 -week  school  and  field  sessions.  Training  emphasis  on  identifi¬ 
cation  and  analysis  of  industrial  areas. 

Completed  28 -week  course  in  aerial  geology  at  Portland  Stale 
College  (1955).  Then  worked  for  1.5  years  for  the  State  of  Oregon 
Department  of  Geology,  evaluating  aerial  photographs  for  geologic 
content,  with  particular  emphasis  on  glacial  activity. 

Received  20  weeks  of  aerial  photography  at  Santa  Ana.  California, 
in  19*42.  Had  8  weeks  of  bomb  interpretation  while  in  the  sth  AF 
in  England  in  19*13:  duly:  inspection  of  bomb  damage  inflicted  on 
Germany.  From  1953  to  19(50  worked  in  an  Air  Force  reserve 
squadron  as  a  photointerpreter,  finding  targets  and  assessing 
damage  after  training  strikes. 

Organized  and  taught  photo-mapping  course  in  civil  engineering  at 
Oregon  State  University  for  8  years.  Completed  a  course  of  photo 
mapping  while  in  the  U.  S.  Army.  Has  had  training  in  Multiplex 
and  slereo-eompa  ragraph . 

Completed  a  *1  -month  photo  interpretation  course  at  Ft.  Sill  while 
in  the  U.S.  Army  in  1951.  Had  short  training  periods  on  target 
evaluation  from  photos  taken  over  1951-1953.  While  in  an  artillery 
unit,  was  a  phoioinlcrp refer  for  purposes  of  selecting  military 
ta  rgets . 


Completed  3 -month  course  in  interpretation  and  use  of  aerial 
photographs  at  the  University  of  Colorado  in  1959.  W  as  an  aerial 
photographer  in  the  Army  and  Air  Force.  Work  included  compila¬ 
tion  of  navigational  charts  from  aerial  photographs  and  photo- 
interpretation . 

Completed  1 2 -week  photogram metry  course  at  University  of  Wash¬ 
ington  in  1951  .  Has  used  aerial  photos  over  the  past  1!)  years  for 
interpretation  purposes  in  timber  appraisal  and  woods  survey.  Has 
worked  with  topographical  mapping  using  stereo  photos  in  conjunction 
with  ground  control  for  contour  mapping. 


Subject 

HS 


AJP 

SHP 

PAH 

EAS 

PLS 

DCO'M 


Training  and  Experience 

Completed  2-month  photointerpreler  course  at  Camp  Ritchie,  Md. , 
in  1942  and  6 -month  photointerpreter  course  in  London  in  1943. 

Was  a  photointerpreter  instructor  at  Camp  Ritchie  for  4  months 
and  had  18  months'  experience  in  World  War  II.  The  interpretation 
consisted  in  planning  invasions,  evaluations  of  enemy  installations, 
bomb-damage  assessment  for  artillery,  and  air  strikes.  In  Korea, 
was  an  interpretation  officer  for  16  months. 

Completed  photointerpretation  course  at  Ft.  Davis  and  Ft.  Bragg, 
for  4  weeks.  Was  an  instructor  in  Basic  Training  Course  (A A  and 
AT)  for  the  13th  Airborne  Division  in  1944-1945. 

Received  6  months'  training  in  topographical  surveying  at  Ft.  Bclvoir 
while  in  the  Army  in  1954.  He  had  experience  in  photography  work 
when  in  the  army  and  when  employed  with  Aero  Service  in 
Philadelphia. 

Aerial  photos  utilized  in  courses  and  thesis  (M.S. )  in  geology. 

Has  had  3  years  of  interpretation  of  geological  features,  including 
stereoscopic  and  photogrammetric  methods. 

Received  formal  photointerpretation  training  in  France  in  1944 
while  in  the  U.S.  Army.  Used  photos  for  briefings  for  airborne 
landings  in  identification  of  terrain  features  and  enemy  positions. 

No  formal  training;  however,  extensive  experience  in  all  phases  of 
aerial  photography  as  a  member  of  Ohio  and  Maryland  National 
Guard  Photo  Sections.  Emphasis  on  scaling  and  map-making,  with 
considerable  work  on  oblique  aerial  photos. 

Was  aerial  photographer  for  the  Northern  Pacific  Railroad.  Worked 
on  land  photos  with  emphasis  on  timber  and  right-of-ways.  Some 
work  was  done  with  infrared  photos.  Developed  and  interpreted 
photos  using  stereo  apparatus. 


APPENDIX  II 

CHARACTERISTICS  OF  TARGET  CHANGES  IN  AERIAL 
PHOTOGRAPHS  USED  AS  STIMULI  IN  EXPERIMENT  I 
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APPENDIX  111 


AERIAL  PHOTOGRAPHS  REPRESENTATIVE  OF  COMPARATIVE 
COVERAGES  USED  IN  EXPERIMENT  II 


Location  Key:  See  attached  rrtap,  this  appendix. 
Used  in  i:10,  QCQ  scale  photographs. 

Usee  in  1:20,000  scale  photographs. 

Used  *n  1:40,000  scale  photographs. 


1 9B-M294  V  42-47 


1  Portsmouth,  N.H.,  Naval  Shipyard 

2  Norfolk,  Va.,  Naval  Air  Station 

3  Brooklyn,  N,  Y . ,  Naval  Shipyard 

4  Little  Creek,  Va,,  Naval  Amphib,  Base 

5  Ft.  Knox,  Ky.,  Army  Depot 

6  Aberdeen  Proving  Grounds,  M.D.,  Army  Depot 

7  Ft.  Bragg,  N.C.,  Army  Depot 

8  Ft,  Eustis,  Va,,  Army  Depot 

9  Ft,  Warren,  Wya.,  Training  Area 

10  Lockbourne  AFB,  Ohio 

11  Loring  AFB,  Maine 

12  Bunker  Hill  AFB,  Indiana 

13  Custer  AFS,  Michigan 

14  Montaulk  AFS,  N.  Y. 

15  Syracuse,  N.  Y.,  Industrial  Facilities 

16  Norfolk,  Va.,  Industrial  Facilities 

17  Newport  News,  Va.,  Industrial  Facilities 


Figure  1 1 1-1  LOCATION  KEY 
Comparative  Coverage  Aerial  Photographs 
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METHOD  OF  DEGRADING  IMAGERY  WITH  GAUSSIAN  FILTER 


APPENDIX  IV 

METHOD  OF  DEGRADING  IMAGERY  WITH  GAUSSIAN  FILTER 


The  technique  that  permitted  the  greatest  manipulation  of  edge  gradients  without 
unwanted  discontinuities  was  what  the  authors  called  the  Fry  technique.  Dr, 
Glenn  A.  Fry  has  published  a  series  of  articles  on  the  method  and  the  advantages 
of  its  use  (Fry,  1957  and  19G1). 

The  technique  uses  a  neutral  density  filter  with  a  density  distribution  that  is  a 
normal  gaussian  or  bell-shaped  curve  along  any  diameter.  The  modal  density 
is  in  the  center  and  decreases  peripherally.  The  size  of  the  filter  is  determined 
by  the  diameter  of  the  entrance  pupil  of  the  lens  with  which  it  is  used.  The 
purpose  of  this  diffusion  filter  is  to  modify  the  intensity  of  the  light  rays  entering 
or  leaving  the  objective  lens.  The  less  accurately  refracted  peripheral  rays 
contribute  proportionately  more  to  the  formation  of  the  image  with  this  filter. 

The  resultant  edge  gradients  are  ogival  in  form,  with  the  slope  of  the  linear 
portion  an  inverse  function  of  the  degree  of  defoeusing.  The  slope  and  shoulder 
of  the  gradient  are  smooth,  as  illustrated  in  the  microdensitometer  traces 
shown  in  Figure  15. 

The  apparatus  used  to  make  the  gaussian  filter  is  schematically  represented  in 
Figure  IV-1,  The  light  from  the  source  is  diffused  in  the  condenser  system  to 
ensure  homogeneous  illuminance  at  the  rotatable  slit.  The  slit  should  be  ma¬ 
chined  and  have  a  lenglh-to-widlh  ratio  of  at  least  10  to  1.  Two  exposures  are 
used  to  produce  the  filter,  with  the  slit  rotated  90  degrees  between  the  first  and 
second  exposure.  The  relative  densities  across  the  filter  vary  as  u  function  o! 
the  distance  between  the  slit  and  the  diffuser,  the  type  of  film,  and  the  develop- 
m e  nt.  p r oc ed u re  s . 

The  size  of  the  gaussian  filter  is  controlled  by  an  adjustment  of  the  distance 
between  the  diffuser  and  the  film  from  which  it  is  made.  The  average  density  of 
the  filler  varies  with  the  length  of  the  exposure,  which  is  controlled  by  a  shutter 
located  at  the  projection  lens. 

This  filter  may  be  used  either  with  the  camera  in  making  a  negative  of  the  pic¬ 
ture  in  which  blur  is  to  be  int  roduced  or  with  a  projector  or  enlarger.  The  lat¬ 
ter  use  is  illustrated  in  Figure  TV-2,  In  either  ease  the  filter  must  be  centered 
on  the  optical  axis  for  the  blur  to  be  homogeneous. 

The  magnitude  of  the  blur  is  produced  by  defocusing  the  projection  equipment, 
and  size  may  be  maintained  by  increasing  or  decreasing  the  projection  distance. 
Quantity  of  blur  may  be  measured  with  an  Air  Force  resolution  chart,  but  must 
be  checked  by  a  microdensitometer  to  determine  if  false  resolutions  have 
occu rred. 
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Figure  IV-1  SCHEMATIC  OF  TECHNIQUE  TO  GENERATE  DIFFUSING  FILTER 

Glenn  A.  Fry  Technique 


LIGHT  SOURCE 


Figure  IV-2  SCHEMATIC  OF  TECHNIQUE  FOR  USING  GAUSSIAN 
FILTER  IN  GENERATING  DIFFERENT  BLUR  QUANTITIES 
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APPENDIX  V 

EXPERIMENT  I  RAW  DATA 


EXPERIMENT  I  RAW  DATA  P.l.  AND  NON-P.l 


CORRECT  IDENTIFICATIONS 


APPENDIX  VI 

EXPERIMENT  II  RAW  DATA 


MV 


cST&h 


PERCENT  TARGETS  DETECTED  EXPERIMENT  II  P.l. 


NUMBER  OF  TARGETS  DETECTED  P.l.  EXPERIMENT  II 


T1 


OTAL  TIMES  (S 
ON  PICTURES  EXP 


1:10,000 

mam 

Picture 

Blur 

Blur 

EE1I 

Em 

No . 

0 

4 

0 

4 

1 

188 

224 

143 

137 

2 

96 

101 

200 

208 

3 

214 

115 

173 

125 

4 

32 

80 

127 

171 

5 

158 

90 

189 

124 

6 

74 

157 

71 

74 

7 

74 

67 

125 

148 

8 

171 

90 

165 

128 

Total 

1007 

924 

1193 

1115 

1:10, 000 

sxs 

Picture 

Blur 

Blur 

Blur 

Blur 

No. 

0 

4 

0 

4 

9 

195 

152 

86 

tEM\ 

10 

115 

86 

208 

252 

11 

82 

134 

77 

85 

12 

47 

34 

178 

148 

13 

87 

180 

119 

no 

14 

162 

40 

78 

70 

15 

142 

91 

184 

209 

16 

145 

134 

94 

66 

Total 

975 

OC1 

KJkJ  1 

1024 

1046 

1:10,000 

AM 

SXS 

Picture 

Blur 

Blur] 

Blur 

BIuT 

No. 

m 

0 

4 

17 

262 

94 

145 

192 

18 

60 

68 

120 

141 

19 

98 

142 

60 

94 

20 

49 

102 

134 

69 

21 

66 

40 

127 

85 

22 

120 

97 

100 

136 

23 

42 

102 

39 

194 

24 

43 

229 

112 

60 

Total 

74 0~^ 

874 

m  7 

971 

APPENDIX  VII 

EXPERIMENT  III  RAW  DATA 


l(il 


EXPERIMENT  III  RAW  DATA  P.l.  AND  NON-P.  I 


COMMISSION  ERRORS 


NON-P. I. 


Note:  Totol  possible  =  8  per  subject  per  condition 


**  Note:  2  photos  per  subject  per  condition 
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